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TRAINING  RESEARCH  IN  AUTOMATED  INSTRUCTION  (TRAIN) 

Space  Fortress:  The  Effects  of  Competition,  Skill,  and  Anxiety  on  Task  Performance 


Scientists:  Regian  and  Shebilske 

Instructors:  Connolly-Gomez  and  Robertson-Schiile 

Total  Subjects:  N  =  153 

Purpose  of  Study:  Tbe  experiment  was  designed  to  examine  tbe  effects  of  competition  on 

computer-based  instruction  for  trainees  with  bigb  to  low  interaction  skill  and 
anxiety.  Subjects  were  taugbt  to  play  Space  Fortress  and  randomly  assigned  to 
groups  of  (1)  no  competition  (lottery  determined  winner  of  $100.00),  (2) 
competition  given  early,  or  (3)  competition  given  late  in  both  (early  or  late 
competition  groups,  bigbest  score  won  $100.00)  In  addition,  individual 
difference  measures  were  taken  of  skill  level  (high  vs.  low)  and  interaction 
anxiety  (high  vs.  low).  There  are  implications  for  improving  performance  in  a 
cost-efficient  manner  by  providing  competition  between  trainees. 

Results  of  Study:  Overall,  subjects  in  Dyad  and  Tetrad  training  groups  learned  as  quickly  as 
subjects  in  individual  training  groups.  In  line  with  Social  Learning  Theory, 
arousal  facilitated  dominant  responses.  Thus,  early  in  training,  dominant 
responses  are  incorrect,  but  later  in  training  dominant  responses  are  correct. 
Competition  enhanced  performance  for  trainees  with  high  skill  level  and 
appears  to  have  no  effect  on  performance  with  low  skill  level,  although  the 
screening  procedure  eliminated  those  trainees  with  the  lowest  skill  levels. 
Subjects  high  in  interaction  anxiety  performed  better  in  Individual  protocols, 
whereas  subjects  who  have  low  interaction  anxiety  performed  better  in  Dyad 
protocols. 

Selected  Publications: 

Arthur,  W.,  Jr.,  Young,  B.,  Jordan,  J.A.,  &  Shebilske,  W.L.  (1996).  Effectiveness  of  dyadic 

training  protocols:  The  influence  of  training  interaction  anxiety.  Journal  of  the  Human 
Factors  Society,  38,  79-86. 

Shebilske,  W.  L.,  Regian,  J.  W.,  &  Jordan,  J.  (1992).  A  dyadic  protocol  for  training  complex 
skills.  Human  Factors,  34,  369-374. 

Worchel,  S.,  Shebilske,  W.L.,  Jordan,  J.A.,  &  Prislin,  R.  (1997).  Competition  and  performance 
on  a  computer-based  complex  perceptual-motor  task.  Human  Factors,  39,  410 

Keywords:  Space  Fortress,  competition,  individual,  dyad,  tetrad,  interaction  anxiety 


1 


The  Console  Maze:  Learning  with  2-D  vs.  3-D  Tasks 


Scientists;  Regian,  Shebilske,  and  Monk 

Instructors:  Connolly-Gomez,  Robertson-Schiile,  and  Monk 

Total  Subjects:  N  =  3 1 

Purpose  of  Study:  To  examine  how  subjeets  learn  a  2-dimensional  console  maze  in  a  computer- 
based  program  and  to  compare  the  results  to  those  obtained  using  a  3- 
dimensional  layout  in  virtual  reality.  Issues  such  as  simulation  fidelity  and 
God’s  eye-view  perspective  were  examined. 

Results  of  Study:  Subjects  learned  spatial-procedural  and  spatial-navigational  skills.  They 
continued  to  acquire  skill  while  being  tested  on  the  procedure,  as  the  tests 
provided  continued  practice  in  executing  the  procedure.  The  maze  data 
showed  that  the  subjects  were  able  to  learn  the  virtual  maze,  as  evidenced  by 
their  ability  to  navigate  within  the  maze  after  training. 

Selected  Publications: 

Regian,  J.W.,  Shebilske,  W.L.,  &  Monk,  J.  M.  (1992).  Virtual  reality:  An  instructional  medium 
for  visual  spatial  tasks.  Journal  of  Communication,  Autumn,  42{A). 

Regian,  J.W.,  Shebilske,  W.L.,  &  Monk,  J.M.  (1992).  A  preliminary  empirical  evaluation  of 
virtual  reality  as  an  instructional  medium  for  visual-spatial  tasks.  Journal  of 
Communication,  42{A),  136-149. 

Keywords:  Space  Fortress,  2-dimensional  layout,  maze,  spatial,  fidelity 


Space  Fortress:  Does  Observational  Learning  Affect  Performance? 

Scientists:  Regian  and  Shebilske 

Instructors:  Connolly-Gomez  and  Robertson-Schiile 

Total  Subjects:  N  =  78 

Purpose  of  Study:  To  determine  the  effect  of  passive  observation  on  Space  Fortress 

performance.  Observers  in  the  No  Observation  Tetrad  did  not  observe  players 
during  game,  but  did  filler  tasks.  This  experiment  has  implications  for 
isolating  observation  factors  as  opposed  to  distribution  of  practice.  Tetrad  and 
No  Observation  Tetrad  have  same  distribution,  but  different  levels  of 
observation. 
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Results  of  Study:  Individual,  Dyad,  and  Tetrad  protocols  replicated  results  from  previous 

studies.  The  No  Observation  Tetrad  performed  the  same  protocol  as  the  Tetrad 
subjects  in  Space  Fortress.  Therefore,  distribution  of  practice  rather  than 
observation  facilitates  performance  in  Tetrad  protocol.  However,  there  was 
not  sufficient  experimental  control  to  completely  restrict  observation. 
Replication  of  the  results  is  needed  before  submission  for  publication 

Selected  Publications:  Awaiting  follow-on  study. 

Keywords:  Space  Fortress,  no  observation,  spatial,  cooperative  learning,  individual,  dyad, 

tetrad 


Phoenix:  Effects  of  Interference  on  Acquisition  of  Automaticity 

Scientists:  Goettl  and  Regian 

Instructors:  Connolly-Gomez  and  Robertson-Schiile 

Total  Subjects:  N  =  104 

Purpose  of  Study:  Subjects  performed  various  single  and  dual  tasks  while  using  the  Phoenix 
desktop-flight  simulator  to  determine  quantitative  and  qualitative  aspects  of 
interference  on  cognitive  demands.  Tasks  included  a  tracking  task  (highway- 
in-the-sky)  and  a  limerick  task.  Performance  measures  and  subjective 
workload  measures  were  hypothesized  to  indicate  both  the  overall  interference 
between  tasks,  and  the  specific  nature  of  the  interference.  The  goal  is  to 
develop  a  battery  of  primary  and  secondary  tasks  to  be  used  for  future  studies 
on  the  acquisition  of  automaticity.  Length  of  the  study  was  approximately  6 
to  8  hours. 

Results  of  Study:  This  experiment  served  as  a  pilot  study  for  the  acquisition  of  the  Phoenix  task. 
Limerick  interfered  minimally  with  Phoenix. 

Selected  Publications:  None. 

Keywords:  Spatial,  secondary  task,  automaticity 
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Cooperative  Algebra  Word  Problem-Solving  Pilot 


Scientists:  Yadrick  and  Regian 

Instructors:  Connolly-Gomez  and  Robertson-Schiile 

Total  Subjects:  N  =  20 

Purpose  of  Study:  To  pilot  different  problem  sets  with  varying  levels  of  diffieulty,  instructions, 
and  procedures.  Subjects  were  tested  every  week  for  a  maximum  of  one  day 
and  a  2-hour  slot  minimum.  This  was  a  pilot  for  a  later  study. 

Results  of  Study:  The  study  assisted  in  normalizing  problems  and  established  difficulty  level. 

Selected  Publications:  None. 

Keywords:  Algebra  word  problems,  quantitative 


Space  Fortress:  Tbe  Effects  of  Discussion,  Competition,  and  Acquisition  on  Performance 

Scientists:  Regian  and  Shebilske 

Instructors:  Connolly-Gomez  and  Robertson-Schiile 

Total  Subjects:  N  =  159 

Purpose  of  Study:  An  earlier  Space  Fortress  competition  experiment  was  replicated  with 
videotaped  group  discussions  added.  The  emphasis  of  the  study  was  to 
determine  the  effects  of  group  discussion  on  Space  Fortress  performance  with 
subjects  in  Individual  training  protocol.  There  are  implications  for  peer- 
tutoring  and  cooperative  learning  contexts  during  training.  This  study  is 
linked  with  a  later  discussion  study. 

Results  of  Study:  Subjects  that  were  trained  under  the  Individual  in  Discussion  protocol 

performed  significantly  better  on  the  Space  Fortress  task  than  subjects  trained 
under  the  Individual  protocol.  The  results  suggest  that  a  relatively  inexpensive 
manipulation,  such  as  group  discussion  can  improve  performance  on  complex 
skill  acquisition. 

Selected  Publications: 

Prislin,  R.,  Jordan,  J.A.,  Worchel,  S.,  Semmer,  F.T.,  &  Shebilske,  W.L.,  (1996).  The  effects  of 
group  discussion  on  acquisition  of  complex  skills.  Human  Factors,  38,  404 
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Keywords:  Space  Fortress  eompetition  and  individual,  dyad,  tetrad  in  discussion 


Space  Fortress:  Do  Training  Differences  Influence  Resistance 
to  Secondary  Task  Interference? 

Scientists:  Regian  and  Shebilske 

Instructors:  Connolly  Gomez  and  Robertson-Sehiile 

Total  Subjects:  N  =  78 

Purpose  of  Study:  This  experiment  was  a  partial  replieate  for  tbe  Spaee  Fortress  eompetition 
experiment  (pg.  1)  (no  eompletion  condition).  Transfer  and  seeondary  tasks 
were  added  after  luneb  on  tbe  second  day  in  place  of  tbe  2-dimensional 
eonsole/maze.  Tbe  seeondary  tasks  were:  keyboard,  limeriek,  tapping,  proteet 
tbe  fortress,  and  a  hybrid  version  of  the  Gopher  version  of  Spaee  Fortress. 

The  purpose  was  to  determine  if  subjects  who  learn  Spaee  Fortress  under 
different  training  protoeols  are  more  resistant  or  less  resistant  to  secondary 
task  interference  and  how  effeetively  they  are  able  to  transfer  skills  eompared 
to  their  eounterparts. 

Results  of  Study:  We  found  that  the  advantages  of  using  dyad  and  tetrad  protoeols  were 
maintained  throughout  the  seeondary  and  transfer  tasks. 

Selected  Publications: 

Shebilske,  W.L.,  Jordan,  J.A.,  Arthur,  W.,  &  Regian,  J.W.  (1993).  Combining  a  multiple 
emphasis  on  eomponents  protoeols  for  training  eomplex  skills.  Proceedings  of  the 
Human  Factors  and  Ergonomics  Society  3  7th  Annual  Meeting  (1212-1 220).  Santa 
Moniea,  CA:  The  Human  Faetors  and  Ergonomies  Soeiety. 

Keywords:  Spaee  Fortress  Transfer,  seeondary  tasks,  spatial,  eooperative  learning,  individual, 

dyad,  tetrad 
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Space  Fortress:  Comparison  of  Focus  on  Component  Scores 
vs  Focus  on  Total  Score  on  Performance? 

Scientists:  Shebilske  and  Regian 

Instructors:  Connolly  Gomez  and  Robertson-Schiile 

Total  Subjects:  N  =  83 

Purpose  of  Study:  Tbe  goal  was  to  determine  tbe  effeets  that  Gopber's  Multiple  Emphasis  on 
Components  (MEC)  paradigm  would  have  on  subjeets  who  learned  Spaee 
Eortress  under  different  training  protoeols.  Subjeets  were  instrueted  to  either 
foeus  on  eomponents  of  total  seore  (e.g.,  eontrol  of  ship,  speed  of  ship,  mines, 
ete.)  to  raise  their  seore,  or  on  total  seore  (a  eombination  of  all  those  elements) 
to  raise  their  seores.  This  has  implieations  for  eombining  Aetive  Interloeked 
Modeling  (AIM)  with  MEC  to  ereate  new,  more  effeetive  training  protoeols. 

Results  of  Study:  Gopher's  results  were  replieated  as  there  was  an  inerease  in  performanee  with 
subjeets  trained  under  the  Individual  protoeol  whieh  foeused  on  the  Multi 
Emphasis  Components  (MEC).  It  was  also  found  that  MEC  eombined  with 
Dyad  protoeol,  but  not  Tetrad  protoeol,  produeed  more  effieient  training. 

Selected  Publications: 

Shebilske,  W.E.,  Jordan,  J.A.,  Arthur,  W.,  &  Regian,  J.W.  (1993).  Combining  a  multiple 
emphasis  on  eomponents  protoeols  for  training  eomplex  skills.  Proceedings  of  the 
Human  Factors  and  Ergonomics  Society  3  7th  Annual  Meeting  (1212-1 220).  Santa 
Moniea,  CA;  The  Human  Eaetors  and  Ergonomies  Soeiety. 


Keywords:  Spaee  Eortress,  multi-emphasis,  individual,  dyad,  tetrad 


Variability  of  Training  and  its  Influence  on  Performance  after  Four  Hours 
Scientists:  Goettl  and  Regian 

Instructors:  Connolly  Gomez  and  Robertson-Sehiile 

Total  Subjects:  N  =  70 

Purpose  of  Study:  This  study  examined  the  effeets  of  variability  of  preeision  during  aequisition 
in  a  four-group,  between-subjeets  experimental  design.  The  variable 
manipulated  for  this  study  ineluded  the  size  of  the  target  gates  (large  or  small) 
during  aequisition,  transfer,  and  re-aequisition.  It  was  hypothesized  that 
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variability  during  acquisition  may  be  more  important  in  transfer.  The  length  of 
the  study  was  approximately  4-6  hours. 

Results  of  Study:  Results  indieated  that  ineluding  a  single,  small  gate  early  in  the  practice  phase 
had  an  immediate  negative  effect  on  performance,  but  a  transient  positive 
effeet  on  later  trials. 

Selected  Publications:  None. 

Keywords:  Spatial,  acquisition,  variability,  transfer 


The  Influence  of  Graphic  Representation  and  Written  Cues 
on  Learning  a  Procedural  Task 

Scientists:  Farquhar  and  Regian 

Instructors:  Connolly  Gomez  and  Robertson-Sehiile 

OTotal  Subjects:  N=153 

Purpose  of  Study:  To  determine  whether  or  not  graphic  representation,  written  cues,  or  both 
assisted  in  learning  a  procedural  task. 

Results  of  Study:  It  was  found  that  a  graphical  model  aided  in  the  acquisition  of  the  procedural 
task. 

Selected  Publications: 

Farquhar,  J.D.,  &  Regian,  J.W.  (1993).  The  effects  of  a  dynamic  graphical  model  during 

simulation-based  training  of  console  operation  skill.  Proceedings  of  the  1993  Conference 
on  Intelligent  Computer-Aided  Training  and  Virtual  Environment  Technology. 

Farquhar,  J.D.  (1992).  A  preliminary  investigation  of  the  acquisition  of  console-operation  skill 
through  a  dynamic  graphical  model  (Technical  report  submitted  to  the  Air  Force  Office  of 
Scientific  Research  Summer  Research  Program). 

Keywords:  Procedural  tasks,  graphics,  learning  aids 


Space  Fortress:  Verification  of  Old  vs.  New  Task 
Scientists:  Shebilske  and  Regian 

Instructors:  Connolly  Gomez  and  Robertson-Sehiile 
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Total  Subjects:  N  =  17 


Purpose  of  Study:  To  determine  if  Spaee  Fortress  performanee  of  subjeets  trained  under  the 
original  Gopher  version  differed  from  those  subjeets  trained  under  the  new 
(Galaxy)  version  of  Spaee  Fortress.  This  will  determine  if  results  from  past 
studies  ean  be  eompared  to  those  of  future  studies. 

Results  of  Study:  Results  suggest  that  there  are  no  signifieant  differenees  between  the  (Galaxy) 
version  and  the  Gopher  version  of  Spaee  Fortress.  Therefore,  previous  and 
future  studies  using  these  different  versions  may  be  eompared  without 
eoneern. 

Selected  Publications:  None. 

Keywords:  Spaee  Fortress,  generalizability,  individuals 


Phoenix:  The  Influence  of  Full  Task  Training  vs.  Component  Training  I 
Scientists:  Goettl  and  Regian 

Instructors:  Connolly  and  Robertson-Sehiile 

Total  Subjects:  N  =  83 

Purpose  of  Study:  This  study  represented  the  first  attempt  of  examining  whole-/eomponent- 
task  training  issues  for  the  Phoenix  desktop-flight  simulator.  Subjeets  were 
randomly  assigned  to  one  of  two  groups  (full-task  training  or  eomponent  task 
training),  aequisition  was  examined  between  groups  that  had  reeeived  training 
either  presented  as  a  whole  task  or  deeomposed  as  elements  of  the  task.  Task 
elements  were  presented  in  a  bloeked  sehedule  during  the  training  phase.  All 
subjeets  reeeived  a  retention  test  using  the  whole  task  rather  than  the 
deeomposed  task.  A  revision  of  this  pilot  experiment  was  eondueted  later 
using  three  treatment  groups  with  two  levels  of  the  eomponent  training 
variable.  The  length  of  the  study  was  16  hours. 

Results  of  Study:  A  faetor  analysis  revealed  several  eomponent  skills  important  to  Phoenix. 

One  faetor,  "gate  aiming"  was  important  early  in  training.  Another  faetor, 
"Spatial  Orientation"  was  important  early  and  late  in  training.  This  study 
suggests  the  utility  of  baekward  transfer  teehnique. 


Selected  Publications: 


Goettl,  B.P.  (1993).  Analysis  of  a  skill  on  a  flight  simulator:  Implications  for  training. 
Proceedings  of  the  Human  Factors  and  Ergonomics  Society  37  th  Annual  Meeting  (1257-1261). 
Santa  Monica,  CA:  The  Human  Factors  and  Ergonomics  Society. 

Keywords:  Spatial,  whole/part  training,  transfer,  aequisition,  eomponents,  faetor  analysis 


Space  Fortress:  Comparison  of  Discussion  vs.  Individual  Training  on  Performance 

Scientists:  Shebilske  and  Regian 

Instructors:  Connolly  and  Robertson-Sehule 

Total  Subjects:  N  =  103 

Purpose  of  Study:  To  determine  the  effeets  of  group  diseussion  on  Spaee  Fortress  performanee 
of  subjeets  in  Individual  training  protoeol.  This  has  implieations  for  peer- 
tutoring  and  eooperative  learning  eontexts  during  training.  This  was  an 
extension  of  the  previous  Diseussion  study. 

Results  of  Study:  Subjects  that  were  trained  under  the  Individual  in  Discussion  protocol 

performed  significantly  better  on  the  Space  Fortress  task  than  subjects  trained 
under  the  Individual  protocol.  The  results  suggest  that  a  relatively 
inexpensive  manipulation  such  as  group  discussion  can  improve  performance 
on  complex  skill  acquisition. 

Selected  Publications: 

Prislin,  R.,  Jordan,  J.A.,  Worchel,  S.,  Semmer,  F.T.,  &  Shebilske,  W.F.,  (1996).  The  effects  of 
group  discussion  on  acquisition  of  complex  skills.  Human  Factors,  38,  404 
Prislin,  R.,  Jordan,  J.  A.,  Shebilske,  W.F.,  (In  preparation). 

Keywords:  Space  Fortress,  individual,  individual  in  discussion 


Scientists: 
Instructors: 
Total  Subjects: 


Space  Fortress  —  Dyadic  Training  without  Social  Contact 

Shebilske  and  Regian 
Connolly  and  Robertson-Schiile 
N  =  86 
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Purpose  of  Study:  To  determine  if  the  partner  in  the  Dyad  training  protoeol  eould  be  replaeed 
with  a  "eomputer"  partner.  Also,  to  isolate  and  eontrol  eomponents  (e.g., 
soeial  motivation)  whieh  eontribute  to  the  effeetiveness  of  dyadie  training. 
Subjeets  were  told  that  the  eomputer  was  eontrolling  the  other  half  of  the 
game  when  in  reality  it  was  their  anonymous  partner— another  subject  who 
was  playing  the  other  half  of  the  game.  This  has  implications  for  whether 
social  contact  is  necessary  as  long  as  the  subject  is  able  to  model  the  actions 
and  reactions  of  some  type  of  partner  (even  if  it  is  a  computer). 

Results  of  Study:  Subjects  that  were  trained  under  the  Hidden-Partner  Dyad  performed  as  well 
as  subjects  trained  under  Individual  protocol  and  Dyad  protocol  from  previous 
studies.  This  suggests  that  social  contact  is  not  necessary  as  long  as  modeling 
of  response  can  occur.  However,  training  two  subjects  at  one  terminal  still 
has  a  greater  cost-benefit  ratio. 

Selected  Publications: 

Shebilske,  W.L.,  Regian,  J.W.,  Arthur,  W.,  &  Jordan,  J.  (1992).  A  dyadic  protocol  for 

training  complex  skills.  Human  Factors,  34,  369-374. 

Keywords:  Space  Fortress,  Dyads  without  social  interaction,  spatial 


Investigating  Backward  Transfer  and  Whole  vs.  Component  Task 
Training  on  Knowledge  Acquisition 

Scientists:  Goettl  and  Regian 

Instructors:  Connolly  Gomez  and  Robertson-Schiile 

Total  Subjects:  N  =  60 

Purpose  of  Study:  This  follow-up  study  examined  task  component  vs.  whole  task  training  using 
the  Phoenix  desktop-flight  simulator.  Subjects  were  assigned  to  one  of  three 
groups:  (1)  whole  task  training,  (2)  component  skills  training  with  whole  task 
transfer,  or  (3)  component  skills  training  control.  Acquisition  was  examined 
between  groups  that  had  received  training  either  on  task  components  that  were 
presented  as  a  whole  task  or  decomposed  as  elements  of  the  task.  Both 
transfer  and  knowledge  tasks  were  used  to  assess  acquisition  during  training. 
The  approximate  length  of  the  study  was  16  hours. 

Results  of  Study:  Replication  of  Phoenix:  The  Influence  of  Full  Task  Training  vs.  Component 
Training;  the  factor  analysis  identified  two  component  skills  important  to 
Phoenix.  One  factor,  "gate  aiming,"  was  important  early  in  training.  Another 
factor,  "Spatial  Orientation,"  was  important  early  and  late  in  training.  This 
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study  verifies  the  results  found  in  the  previous  study  and  suggests  the  utility  of 
baekward  transfer  teehnique. 

Selected  Publications:  None. 

Keywords:  Spatial,  whole/part  training,  transfer,  eomponents,  aequisition,  faetor  analysis 


Efficiency  of  Learning:  Dyads  vs.  Individuals  for  Algebra  Word  Problems 

Scientists:  Yadriek  and  Regian 

Instructors:  Connolly  Gomez  and  Robertson-Sehiile 

Total  Subjects:  N  =  30 

Purpose  of  Study:  To  compare  teaming  differences  using  the  unguided  version  of  Mtutor.  The 
subjects  were  placed  in  either  an  individual  or  dyad  condition  using  the 
unguided  version  of  the  Mtutor. 

Results  of  Study:  There  were  no  differences  found  for  dyads  or  individuals.  There  was  no 

learning  effect,  due  to  ceiling  effects  and  minimal  learning  had  taken  place. 

Selected  Publications:  None. 

Keywords:  Algebra  word  problems,  quantitative,  cooperative  learning,  dyads  vs.  individuals 


The  Influence  of  Abbreviated  vs.  Extended  Practice  in  a  Two-year  Longitudinal  Study 

Scientists:  Shute  and  Gawlick-Grendell 

Instructors:  Connolly  and  Robertson-Sehiile 

Total  Subjects:  N  =  92 

Purpose  of  Study:  To  compare  retention  of  information  acquired  from  the  Flight  Engineering 
Tutor  training  acquired  two  years  previously.  In  the  original  study  (1990), 
there  were  different  practice  conditions  where  students  would  receive  either 
three  abbreviated  or  12  extended  practice  problems.  In  addition,  sensitivity  to 
relearning  was  investigated  following  a  2-hour  refresher  course. 

Results  of  Study:  Two  experiments  examined  practice  effects  on  parameters  of  skill  acquisition, 
learning  outcome,  retention,  and  sensitivity  to  re-learning.  In  Experiment  1, 
our  learning  criterion  task  was  an  intelligent  tutoring  system  teaching  flight 
engineering  knowledge  and  skills  with  two  practice  conditions:  “Extended” 
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and  “Abbreviated,”  differing  only  in  the  number  of  practiee  problems 
requiring  solution  across  problem  sets.  Results  from  the  first  experiment 
showed  that  subjects  in  the  Abbreviated  condition  completed  the  curriculum, 
overall,  in  half  the  time  it  took  the  Extended  subjects,  but  at  the  expense  of 
greater  errors  and  latencies  during  problem  solution.  Despite  these  acquisition 
differences,  both  groups  performed  the  same  across  all  learning  outcome 
measures.  Experiment  2  examined  the  benefits  of  extended  practice  on 
retention  and  sensitivity  to  relearning  after  a  significant  period  of  time  had 
elapsed  (2  years).  Although  the  sample  size  was  fairly  small  (34  returning 
subjects),  we  found  significant  group  differences  in  terms  of  retention  of 
previously  learned  material,  hollowing  a  2-hour  “refresher  course,”  there  was 
no  evidence  of  differential  sensitivity  to  re-leaming.  Data  showed  that  the 
benefits  of  initial  extended  practice  emerged. 

Selected  Publications: 

Shute,  V.J.  &  Gawlick-Grendell,  E.A.  (1994,  April).  Practice  effects  on  skill  acquisition, 
outcome,  and  retention.  Paper  presented  at  the  American  Educational  Research 
Association,  New  Orleans,  EA. 

Shute,  V.J.  &  Gawlick  E.A.  (1995).  Practice  effects  on  skill  acquisition,  outcome,  and  retention. 
Human  Factors,  12,  \. 

Keywords:  Plight  Engineering  Tutor,  retention 


Stat  Lady  I  —  Probability  Module  Workbook  vs,  Computer-Based  Tutor 
Scientists:  Shute  and  Gawlick-Grendell 

Instructors:  Connolly  and  Robertson-Schiile 

Total  Subjects:  N  =  107  computer-based,  N  =  104  workbook  version,  N  =  100  no  treatment 
control 

Purpose  of  Study:  The  purpose  of  the  study  was  twofold:  (1)  to  test  the  computer's 

contribution  to  learning  outcome,  efficiency,  and  overall  enjoyment  of  the 
learning  process  and 

(2)  to  compare  the  possible  differences  in  instruction  via  statistics. 

Results  of  Study:  Overall,  there  were  no  significant  differences  between  treatment  conditions 

(e.g.,  Stat  Eady  computer  versus  workbook).  However,  students  placed  in  the 
computer-based  condition  learned  more  declarative  knowledge  in  comparison 
to  students  placed  in  the  workbook  condition.  Aptitude-treatment  interactions 
were  also  investigated,  and  the  results  found  that  high-aptitude  students 
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learned  more  from  the  computer  version  of  the  tutor,  while  low-aptitude 
students  learned  the  same,  regardless  of  treatment  condition. 


Selected  Publications: 

Shute,  V.J.,  Gawlick-Grendell,  L.  A.,  Young,  R.K.,  &  Burnham,  C.A.  (1995).  An  experiential 
system  for  learning  probability:  Stat  Lady  description  and  evaluation.  Instructional 
Science  (4)  1-22. 

Keywords:  Stat  Lady  -  Intro,  to  Probability  Statistics  Tutor,  ITS,  ATI 


Spatial  Learning  in  a  Physical  vs.  Virtual  Environment 

Scientists:  Regian,  Shebilske,  and  Monk 

Instructors:  Monk,  Morlen,  and  Gonzalez 

Total  Subjects:  N  =  22 

Purpose  of  Study:  The  purpose  was  twofold:  (1)  to  investigate  differences  in  spatial  learning 
between  a  VR  environment  and  the  actual  physical  environment,  and  (2)  to 
describe  several  applications  of  Virtual  Reality  technology  in  conjunction 
with  the  tasks  for  the  Federal  Bureau  of  Investigation  (FBI)  during  the  1996 
Olympic  Games. 

Results  of  Study:  The  results  were  that  the  subjects  were  able  to  navigate  through  the  physical 
environment  after  training  on  the  2-D  console  and  maze. 

Selected  Publications: 

Regian,  J.W.  (1986).  An  assessment  procedure  for  configurational  knowledge  of  large-scale 
space.  Unpublished  doctoral  dissertation.  Santa  Barbara  CA:  University  of  California. 

Regian,  J.W.  &  Shute,  V.J.  (1993,  April).  An  investigation  of  virtual  reality  for  training  visual- 
spatial  tasks.  Paper  presented  at  the  American  Educational  Association,  Atlanta,  GA. 

Keywords:  Virtual  reality  versus  Physical,  spatial 

Investigating  How  Different  Learning  Environments  Affect  Exploratory  Behavior 

Scientists:  Shute 

Instructors:  Connolly  and  Robertson-Schiile 

Total  Subjects:  N  =  400 
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Purpose  of  Study:  The  main  question  addressed  whether  there  is  a  main  effect  of  learning 
environment  when  comparing  learning  outcomes  between  discovery  (or 
inductive)  and  directed  kinds  of  instructional  approaches,  furthermore,  can 
aptitude-treatment  interactions  (ATI)  be  replicated  where  some  people  learn 
better  in  discovery  environments  and  others  learn  better  under  different 
conditions?  The  study  investigated  learning  differences  as  a  function  of  being 
assigned  to  either  a  rule-induction  or  rule-application  learning  environment 
using  a  confirmatory  test  of  ATI.  That  is,  subjects  were  placed  a  priori  in  one 
of  two  computerized  learning  environments  based  on  a  decision  rule  obtained 
from  a  previous  study  using  an  electricity  tutor  (Shute,  1993).  The  learning 
style  (or  aptitude)  investigated  was  "exploratory  behavior." 

Results  of  Study:  A  significant  3 -way  interaction  was  found  among  learning  environment, 
exploratory  behavior,  and  math  skill.  Subjects  in  the  “few”  math  courses 
category  clearly  showed  the  predicted  disordinal  interaction  between 
exploratory  behavior  (low  vs.  high)  and  learning  environment  (inductive  vs. 
applied).  However,  subjects  who  had  taken  “many”  math  courses  showed  a 
disordinal  interaction,  but  in  the  opposite  direction.  Implications  of  these 
findings  are  discussed  in  relation  to  macroadaptive  approaches  to  the  design  of 
ITS. 

Selected  Publications: 

Shute,  V.J.  (1992).  Aptitude-treatment  interactions  and  cognitive  skill  diagnosis.  In  J.W. 

Regian  &  V.J.  Shute  (Eds.),  Cognitive  approaches  to  automated  instruction  (pp.1-14). 
Hillsdale,  NJ:  Lawrence  Erlbaum  Associates. 

Shute,  V.J.  (1993).  A  comparison  of  learning  environments:  All  that  glitters. ..In  S.P.  Lajoie  & 
S.J.  Derry  (Eds.),  Computers  as  cognitive  tools  (pp.  47-74),  Hillsdale,  N.J:  Lawrence 
Erlbaum  Associates. 

Shute,  V.J.  (1994).  Learners,  environments,  and  a  confirmatory  test  of  ATI.  Paper  presented  at 
American  Educational  Research  Association,  New  Orleans,  LA. 

Keywords:  Electricity  Tutor,  ITS,  induction  vs.  application,  exploratory  behavior 


Intelligent  Instruction  vs.  Passive  and  Rudimentary  Coaching  in  Knowledge  Acquisition 
Scientists:  Shahidi  and  Regian 

Instructors:  Connolly  and  Robertson-Schiile 

Total  Subjects:  N  =  105 


14 


Purpose  of  Study:  This  study  investigated  the  issue  of  whether  students  learn  better  with  online 
intelligent  coaehing  systems  driven  by  student  modeling  rather  than  with 
passive  online  help  systems.  There  were  three  treatment  groups:  (1)  passive 
online  help,  (2)  intelligent  instruetion  based  on  student  model,  and  (3) 
rudimentary  coaching.  It  was  hypothesized  that  intelligent  instruction  using  a 
student  model  would  train  subjects  better  in  terms  of  achievement  as 
measured  by  the  number  of  steps  to  solve  the  problem,  total  errors,  number  of 
critical  errors,  testing  time  and  time  necessary  to  train.  All  groups  received  3- 
4  hours  of  classroom  instruction  on  how  to  use  an  electronic  spreadsheet  and 
were  then  given  several  practice  problems  and  two  test  problems.  The  total 
length  of  the  study  was  6-8  hours. 

Results  of  Study:  Results  indicated  that  the  intelligent  coaching  instruction  was  significantly 
better  than  the  rudimentary  and  passive  online  coaching  systems. 

Specifically,  students  trained  with  the  intelligent  help  using  the  student  model 
used  fewer  steps  to  complete  the  goal,  committed  fewer  errors,  less  critical 
errors,  took  less  time  to  train  and  test,  and  preferred  that  type  of  instruction 
overall. 

Selected  Publications:  None. 

Keywords:  ITSs,  coaching,  instruction,  student  models,  help-systems,  quantitative 


The  Use  of  Synthetic  Speech  in  Investigating  the  Perceptual  Effects  Within  Training 
Scientists:  Connolly  Gomez,  Gonzalez,  Shebilske,  and  Regian 

Instructors:  Connolly  Gomez  and  Robertson-Schiile 

Total  Subjects:  N  =  133 

Purpose  of  Study:  Previous  studies  have  revealed  that  the  perception  and  comprehension  of 
synthetic  speech  may  be  attributed  to  increased  processing  demands  in  short¬ 
term  memory.  This  increase  in  short-term  memory  demand  is  reflected  in 
serial  recall  and  variable  inter-stimulus  rate  tasks.  Additionally,  it  has  been 
consistently  shown  that  the  perception  of  synthetic  speech  improves  with 
moderate  amounts  of  training.  The  present  study  was  conducted  to  determine 
if  the  increased  perceptual  effects  of  training  are  due  to  a  reduction  of  short¬ 
term  memory  load  as  reflected  in  serial  recall  and  varied  presentation  rate 
recall  tasks.  Two  groups  of  subjects  were  tested  with  synthetic  speech  using 
the  same  recognition  tasks  recognition  before  and  after  training.  One  group 
was  trained  with  synthetic  speech  and  the  other  group  acted  as  the  control, 
receiving  no  training  between  the  pretest  and  posttest  interval.  The 
approximate  length  of  the  study  was  16  hours  over  a  one -week  period. 
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Results  of  Study:  Results  indicated  that  subjects  who  trained  on  synthetic  speech  versus  the 
control  group,  who  received  no  training  between  baseline  and  posttest, 
performed  better  on  short-term  memory  tasks.  The  results  of  this  study 
indicate  that  training  may  have  reduced  some  of  the  cognitive  burden  that 
synthetically  generated  auditory  displays  place  upon  the  user. 

Selected  Publications: 

Connolly  Gomez,  C.A.,  Shebilske,  W.L.,  &  Regian,  J.W.  (1994).  The  effects  of  training  on 

capacity  demands  in  short-term  memory  for  synthetic  speech.  Proceedings  of  the  Human 
Factors  and  Ergonomics  Society  38th  Annual  Meeting  (1229-1233).  Santa  Monica,  CA; 
The  Human  Factors  and  Ergonomics  Society. 

Keywords:  Synthetic  speech,  working  memory,  intelligibility,  capacity  demands 


Phoenix:  Investigating  Effects  of  Blocked  vs.  Varied  Practice 
Scientists:  Goettl  and  Regian 

Instructors:  Connolly  and  Robertson-Schule 

Total  Subjects:  N  =  82 

Purpose  of  Study:  This  study  compared  Carlson's  working  memory  hypothesis  and  Glenberg's 
multiple-components  theory  of  contextual  interference  effect.  Both  theories 
predict  that  blocked  practice  on  the  task  component  will  be  less  effective  than 
varied  practice.  Furthermore,  Glenberg  predicts  that  a  completely  random 
schedule  will  be  more  effective  than  a  schedule  consisting  of  replications  of 
one  sequence.  Subjects  were  randomly  placed  into  one  of  three  treatment 
groups:  (1)  whole-task  training,  (2)  blocked  presented  component  training, 
or  (3)  repeated  sequence  component  training.  Subjects  were  given  a  retention 
task  on  the  whole  task  and  a  transfer  to  another  task.  In  addition,  spatial  and 
attentional  individual  differences  were  assessed  using  CAM,  VS  AT,  and 
Space  Fortress  aiming  task.  Phoenix  testing  phases  always  followed  a  24  hr. 
retention  interval.  The  length  of  the  study  was  approximately  20  hours. 

Results  of  Study:  Contextual  interference  condition  showed  no  benefits  over  blocked  condition. 

The  beneficial  effects  of  mixed  practice  observed  in  simple  tasks  were  not 
obtained  in  this  complex  task.  These  results  challenge  the  generalizability  of 
the  effect. 


16 


Selected  Publications: 


Goettl,  B.P.  (1994).  Contextual  interface  effects  on  acquisition  and  transfer  of  a  complex  motor 
task.  Proceedings  of  the  Human  Factors  and  Ergonomics  Society  38th  Annual  Meeting. 
(1220-1224).  Santa  Monica,  CA:  The  Human  Factors  and  Ergonomics  Society. 


Keywords:  Spatial,  contextual  interference,  massed  practice,  whole/part  training 


The  Effectiveness  of  Computer-Based  Word  Problem  Solving  Environment  (WPSE) 

Compared  to  Paper  &  Pencil  Tests 

Scientists:  Yadrick  and  Regian 

Instructors:  Connolly  and  Robertson-Schule 

Total  Subjects:  N  =  71 

Purpose  of  Study:  The  goal  of  this  study  was  to  compare  problem  solving  skill  in  a  computer- 
based  Word  Problem  Solving  Environment  (WPSE)  with  paper  and  pencil 
tests.  There  were  5  conditions:  (1)  subjects  using  WPSE  only,  (2)  subjects 
using  paper  and  pencil  only  with  worked  examples,  (3)  subjects  using  "paper 
WPSE  ",  (e.g.,  all  problems,  instruction  screens,  hints  and  help  in  booklet 
form),  (4)  subjects  given  paper  and  pencil  problems  and  solutions  (they  had 
to  work  out  the  problem  using  the  WPSE),  and  (5)  subjects  given  similar 
worked  example  problems,  but  had  to  work  and  enter  solutions  into  the 
WPSE. 

Results  of  Study:  Results  are  complex,  but  briefly.  Groups  1  and  5  did  poorly,  whereas  Groups 
2,  3,  4  did  relatively  well. 

Selected  Publications: 

Yadrick,  R.  M.,  Regian,  J.  W.,  Robertson-Schule,  E.,  &  Gomez,  C.  C.  (1996).  Interface, 

instructional  approach,  and  domain  learning  with  a  mathematics  problem-solving  environment. 

Computers  in  Human  Behavior,  12(4),  527-548. 

Keywords:  Algebra  word  problems,  varied  forms  of  instructional  support 
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Feedback:  The  Effect  of  Elaborative  vs.  Corrective  and  Delayed  vs.  Immediate 


Scientists:  Farquhar  and  Regian 

Instructors:  Connolly  and  Robertson-Schiile 

Total  Subjects:  N  =  128 

Purpose  of  Study:  Loader  is  a  complex  problem-solving  task  requiring  the  operation  of  a 

remote  crane  control  arm  to  load  various  canisters  from  a  set  of  storage  bins  to 
one  or  more  railroad  cars  via  a  simulated  control-panel  console.  The 
computer-based  simulation  demonstrates  a  number  of  features  including  an 
on-screen  dynamic  display,  an  expert  system  module,  flexible  feedback 
messages,  and  a  student  module  with  the  ability  to  output  extensive  data  for 
analysis.  The  experiment  was  a  2-variable  design  examining  elaborative 
“intelligenf  versus  corrective  “dumb”  feedback  and  delayed  versus  immediate 
feedback.  The  approximate  length  of  the  study  was  4-6  hours. 

Results  of  Study:  Results  showed  that  treatment  groups  were  equivalent  in  terms  of  acquisition 
time  however,  the  elaborative  feedback  treatment  group  were  25%  faster  and 
40%  more  accurate  than  the  corrective  feedback  group  during  initial  training. 
These  differences  disappeared  during  later  training  stages.  Although  subjects 
indicated  their  preference  for  delayed  feedback  over  immediate  feedback  for 
non-critical  errors,  no  treatment  differences  were  found  suggesting  that  the 
tutor  may  be  just  as  effective  when  feedback  is  provided  for  only  a  limited 
number  of  critical  elements. 

Selected  Publications: 

Farquhar,  J.D.,  &  Regian,  J.W.  (1994).  The  type  and  timing  of  feedback  within  an  intelligent 
console-operations  tutor.  Proceedings  of  the  Human  Factors  and  Ergonomics  Society 
38th  Annual  Meeting  (1225-1228).  Santa  Monica,  CA:  The  Human  Factors  and 
Ergonomics  Society. 

Farquhar,  J.D.  and  Regian,  J.W.  (1994).  Effects  of  knowledge  representation  during  computer- 
based  training  of  console  operation  skill.  Paper  presented  at  the  1994  Conference  of  the 
Association  for  Educational  Communications  and  Technology,  p.  2-15. 

Earquhar,  J.D.  (1993).  The  design,  development,  and  preliminary  evaluation  of  EO AD ER:  A 
tool  for  investigating  the  acquisition  of  console-operation  skill.  Final  Report,  Graduate 
Student  Research  Program,  Armstrong  Laboratory, 'p.  17-1;  17-15. 

Earquhar,  J.D.  (1993,  June).  Acquisition  of  console-operation  skill  through  a  dynamic  graphical 
model.  Insight,  15(2),  5-7. 

Keywords:  Procedural,  instructional  strategies,  feedback,  displays,  simulation 
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Aircraft  Recognition:  The  Effect  of  Random  Distributed  Practice 
for  Long-Term  Training 


Scientists:  Goettl  and  Regian 

Instructors:  Connolly  Gomez  and  Robertson-Schiile 

Total  Subjects:  N  =  97 

Purpose  of  Study:  The  eontextual  interferenee  effeet  has  been  well  doeumented  in  the  literature 
for  reeognition  of  eomplex  visual  stimuli.  However,  many  of  these 
experiments  allowed  only  a  few  seeonds  for  "study"  time  and  tested  reeall 
minutes  after  praetiee.  Furthermore,  random-distributed  praetiee  was  shown 
to  produee  better  reeognition  than  massed  praetiee  when  testing  eonditions 
were  varied  from  praetiee  eonditions.  The  goal  of  this  researeh  is  to  examine 
the  random-distributed  effeet  for  long-term  training.  Subjeets  viewed  images 
of  military  aireraft  in  one  of  four  eonditions:  (1)  massed,  (2)  sueeessive- 
distributed,  (3)  reeursive-distributed,  and  (4)  random-distributed.  During 
eaeh  study-test  phase,  subjeets  viewed  7  target  aireraft  for  8  seeonds  eaeh. 
Targets  were  randomly  tested  and  interspersed  with  13  distraeters.  In 
addition,  half  of  the  targets  and  distraeters  were  shown  in  a  different  format 
(aetual  pietures).  Subjeets  indieated  whether  they  had  seen  the  aireraft  before 
and  rate  how  eonfident  they  were  in  their  answer.  The  approximate  length  of 
the  study  was  2-3  hours. 

Results  of  Study:  Benefits  of  mixed  presentation  was  obtained  eompared  to  massed 
presentations.  However,  the  random  presentation  (high-eontextual 
interferenee)  was  not  superior  to  repeated  sequenee  (low-eontextual 
interferenee).  These  results  ehallenge  Glenberg's  theory  of  multiple 
eomponents  training. 

Selected  Publications: 

Goettl,  B.P.  (1996).  The  spaeing  effeet  in  aireraft  reeognition.  Human  Factors,  3,  1. 

Keywords:  Massed  praetiee,  distributed  praetiee,  visual  stimuli,  eontextual  interferenee 
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Space  Fortress:  Longitudinal  Investigation  of  Knowledge  Decay 
and  Reacquisition  of  Skill 

Scientists:  Regian  and  Shebilske 

Instructors:  Connolly  and  Robertson-Schiile 

Total  Subjects:  N  =  21 

Purpose  of  Study:  Tbe  focus  was  to  determine  tbe  effects  of  a  long  period  of  nonpractiee  (>  6 
months)  on  Spaee  Fortress  skill  retention  and  reaequisition.  This  may  have 
implieations  for  maintenanee  training  in  operational  training  environments 
and  time  required  to  reaequire  previously  learned  skills. 

Results  of  Study:  As  expeeted,  we  found  substantial  forgetting  of  skills,  but  also  substantial 
inerease  in  speed  of  reaequisition  of  skills.  This  study  serves  to  establish  a 
baseline  for  future  studies  as  well  as  provide  important  information  on 
maintenanee  training  of  eomplex  skills. 

Selected  Publications:  None. 

Keywords:  Spaee  Fortress,  retention,  spatial,  individual 


Phoenix:  Gender  Differences  in  Component  Task  Training 
Scientists:  Goettl  and  Regian 

Instructors:  Connolly  and  Robertson-Sehule 

Total  Subjects:  N  =  1 14 

Purpose  of  Study:  A  factor  analysis  of  Phoenix  desktop-flight  simulator  data  (collected  in  the 
component  skills  studies)  revealed  two  major  groups  of  component  skills. 
These  two  groups  of  skills  are  referred  to  as  spatial  orientation  and  gate 
aiming.  Comparison  of  high-ability  and  low-ability  subjects  indicated  that 
high-ability  subjects  were  significantly  better  on  the  spatial  orientation  skills 
before  and  after  slalom  training.  However,  high-ability  subjects  performed 
better  than  low-ability  subjects  on  gate-aiming  skills  before,  but  not  after, 
slalom  training.  Presumably  spatial-orientation  skills  represent  strong 
individual  differences  that  are  difficult  to  train,  but  are  related  to  skilled 
performance  on  Phoenix.  It  is  hypothesized  that  gate-aiming  skill  seems  to  be 
a  task-specific  ability  that  is  easily  learned,  but  not  highly  predictive  of  skilled 
performance  on  Phoenix.  This  study  compares  three  training  programs: 
whole  task  training,  spatial-orientation  training,  and  gate-aiming  training. 
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Training  effectiveness  will  be  assessed  using  a  part-task  paradigm.  The  length 
of  the  study  was  approximately  16  hours. 


Results  of  Study:  This  study  directly  tested  the  importance  of  these  tasks  using  a  part-task 
training  paradigm.  One  group  of  participants  received  part-task  training 
concentrating  on  the  critical  component  tasks.  A  second  group  received  part- 
task  training  on  non-critical  component  tasks.  The  third  group  received 
whole-task  training.  Participants  receiving  part-task  training  on  the  critical 
tasks  performed  as  well  as  participants  in  the  whole-task  condition.  Females, 
but  not  males,  receiving  practice  on  the  “non-critical”  tasks  performed  worse 
than  females  in  the  other  two  groups.  This  suggests  an  aptitude  by  treatment 
interaction.  Results  were  not  replicated  on  a  transfer  task.  These  data 
illustrate  the  importance  of  selecting  component  tasks  for  part-task  training, 
and  the  utility  of  the  backward-transfer  technique  in  identifying  those  tasks. 

Selected  Publications: 

Goettl,  B.P.  (1995).  Part-task  training  of  complex  tasks:  Utility  of  backward  transfer. 

Proceedings  of  the  39th  Annual  Meeting  of  the  Human  Factors  &  Ergonomics  Society 
(1345-1349).  Santa  Monica,  CA:  The  Human  Factors  &  Ergonomics  Society. 

Keywords:  Spatial  skill,  aptitudes,  factor  analysis,  whole/part  training 


Space  Fortress:  The  Effects  of  Training  with  Variable  vs. 

Multiple  Emphasis  on  Components 

Scientists:  Shebilske,  Corrington,  Jordan,  and  Regian 

Instructors:  Connolly  and  Robertson-Schiile 

Total  Subjects:  N  =  1 18 

Purpose  of  Study:  Literature  on  motor  skill  learning  indicates  that  emphasizing  multiple 

components  over  trials  results  in  higher  acquisition  and  performance.  This 
finding  is  somewhat  analogous  to  the  part/whole  training  theory.  This 
suggests  that  training  improves  when  the  task  components  are  practiced 
individually  rather  than  the  entire  task  practiced  as  a  whole.  In  this  study,  a 
factorial  combination  of  two  variables,  (Individuals  versus  Dyads)  and 
(Multiple  Emphasis  on  Components-MEC  versus  Variable  Emphasis  on 
Components- VEC)  created  4  between-subject  groups.  Both  MEC  and  VEC 
treatment  groups  receive  a  total  of  16  emphasis  for  each  component  score, 
however  the  MEC  treatment  group  receives  the  emphasis  on  successive  game 
sessions.  Conversely,  the  VEC  group  receives  emphasis  instructions  that  vary 
within  game  sessions.  In  order  to  enhance  the  potential  immunization  against 
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habituation  in  massed  trials,  every  adjaeent  trial  within  a  praetiee  session  has  a 
different  emphasis.  The  approximate  length  of  the  experiment  was  24  hours. 

Results  of  Study:  It  is  hypothesized  that  VEC  will  share  MEC's  advantages  of  part  practice 
within  the  whole  task  learning  and  will  have  the  added  advantage  of 
immunizing  against  degraded  performance  in  massed  practice.  VEC  should 
also  combine  effectively  with  dyadic  protocol,  which  would  make  VEC- 
Dyadic  protocol  the  most  effective  to  date  for  training  Space  Eortress.  Initial 
results  however,  do  not  support  this  hypothesis. 

Selected  Publications:  Eollow-up  study  in  preparation. 

Shebilske,  W.E.,  Jordan,  J.,  Arthur,  W.,  &  Regian,  J.W.  (1993).  Combining  a  multiple  emphasis 
on  components  with  small  group  protocols  for  training  complex  skills.  Proceedings  of 
the  Human  Factors  and  Ergonomics  Society,  37th  Annual  Meeting  (1215-1219).  Santa 
Monica,  CA:  The  Human  Eactors  and  Ergonomics  Society. 

Keywords:  Variable  emphasis,  multiple  emphasis,  whole/part  training,  dyadic 


Algebra  Word  Problems  (AWP)  Dyads  vs.  Individuals  Passive  and  Interactive 

Scientists:  Yadrick  and  Regian 

Instructors:  Connolly  and  Robertson-Schule 

Total  Subjects:  N  =  68 

Purpose  of  Study:  This  was  a  2  X  2  factorial  design  where  performance  differences  were 

compared  in  solving  problems  using  a  passive  version  of  a  math  tutor  (WPSE) 
versus  an  active  version  (Solver).  Subjects  were  assigned  to  individual  or 
dyad  protocol. 

Results  of  Study:  Individuals  showed  substantial  improvement  with  the  WPSE  system,  but  no 
improvement  with  the  Solver  system.  Dyads  showed  no  main  effects 
regarding  version;  however,  they  showed  moderate  improvement  from  pre  to 
posttest  with  both  versions. 

Selected  Publications: 

Yadrick,  R.  M.,  Regian,  J.  W.,  Robertson-Schule,  E.,  &  Gomez,  C.  C.  (1996).  Interface, 
instructional  approach,  and  domain  learning  with  a  mathematics  problem-solving 
environment.  Computers  in  Human  Behavior,  12{4),  527-548. 
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Yadrick,  R.M.,  Regian,  J.W.,  Connolly-Gomez,  C.,  &  Robertson-Schiile,  L.  (1997).  Dyadic  vs. 
individual  practice  with  exploratory  and  direetive  mathematies  tutors.  Journal  of 
Educational  Computing  Research,  77(2),  165-186 

Keywords:  Algebra  Word  Problems,  dyads  vs.  individuals 


Phoenix:  Observational  vs.  Hands-on  Learning 
Scientists:  Goettl,  Connolly  and  Regian 

Instructors:  Connolly  and  Robertson-Schiile 

Total  Subjects:  N  =  88 

Purpose  of  Study:  Examined  which  aspects  of  a  complex,  psychomotor  task  necessary  for  the 
Phoenix  desktop-flight  simulator  can  be  learned  through  observation  and  what 
the  optimal  ratio  of  observation  to  hands-on  practice  is.  Subjects  were 
assigned  to  one  of  five  treatment  groups:  (1)  perform  slalom  and  strike,  (2) 
observe  slalom  and  strike,  (3)  perform  slalom  and  observe  strike,  (4)  observe 
slalom  and  perform  strike,  and  (5)  filler  task  control.  The  filler  task  consisted 
of  playing  Windows  Solitaire.  Questionnaires  were  administered  at  several 
points  throughout  the  training  phase  in  order  to  assess  subjects'  knowledge  of 
and  strategies  for  Phoenix.  The  length  of  the  study  was  approximately  16 
hours. 

Results  of  Study:  This  study  tested  the  prediction  that  observational  learning  will  be  more 

effective  for  motor  tasks  having  substantial  cognitive  demands  than  for  those 
that  do  not.  Subjects  were  divided  into  three  treatment  groups:  performers, 
observers,  and  no-observe  controls.  In  Phase  I,  subjects  were  trained  on  a 
computer-based  flight  task  requiring  relatively  little  cognitive  demands.  In 
Phase  11,  subjects  were  trained  on  a  different  flight  task  that  had  significant 
cognitive  and  strategic  demands.  In  Phase  1,  performers  were  superior  to  both 
observers  and  controls;  the  observers  did  not  differ  significantly  from  the 
controls.  In  Phase  11,  observation  showed  a  beneficial  effect  for  females.  The 
female  observers  performed  as  well  as  the  female  performers.  The  results  of 
this  study  suggest  that  observational  learning  benefits  tasks  with  significant 
cognitive  components  more  than  tasks  that  are  primarily  psychomotor. 
Implications  for  computer-based  training  are  discussed. 

Selected  Publications: 

Goettl,  B.P.,  &  Gomez,  C.C.  (1995).  The  role  of  observational  learning  in  automated 
instruction  of  complex  tasks.  Proceedings  of  the  39th  Annual  Meeting  of  the  Human  Factors  & 
Ergonomics  Society  (1335-1339).  Santa  Monica,  CA:  The  Human  Factors  &  Ergonomics 
Society. 
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Keywords:  Spatial,  observation,  cognitive  strategies,  modeling 


ACTIVE  vs,  MITT:  Preference  of  Non-Gaming  vs.  Gaming  Instruction 

Scientists:  Connolly,  Norton,  Gonzalez,  Kline,  Johnson,  and  Robertson-Schiile 

Instructors:  Connolly  and  Robertson-Schule 

Total  Subjects:  N  =  73 

Purpose  of  Study:  The  goal  of  the  study  was  to  assess  the  user  preferences  and  opinions, 
difference  in  context-specific  knowledge  between  users,  and  the  value  of 
gaming  versus  non-gaming  computer-based  training.  Participants  were 
randomly  assigned  to  two  treatment  groups;  MITT  (non-gaming)  or  ACTIVE 
(gaming).  Participants  in  each  treatment  group  were  instructed  for 
approximately  3  hours  and  then  given  a  posttest  and  an  attitude  questionnaire. 
Following  the  posttest  participants  were  instructed  on  the  opposite  tutor  for  1 
hour.  Participants  were  then  given  a  second  posttest  and  attitudinal 
questionnaire.  Participants  were  not  used  in  the  study  if  they  rated  themselves 
as  experts  in  automobile  maintenance  or  had  never  driven  a  car.  The  length  of 
the  study  was  approximately  6-8  hours. 

Results  of  Study:  No  statistically  significant  differences  were  found  for  automotive  maintenance 
knowledge  between  treatment  groups  following  either  poshest  despite  almost 
1  hour  less  time  on  training  for  the  gaming  environment.  However, 
approximately  83%  of  items  were  ranked  higher  by  gaming  participants  on  the 
first  attitude  questionnaire.  After  training  in  both  environments  a  majority  of 
participants  preferred  and  felt  they  learned  more  in  the  gaming  environment. 

In  addition,  all  levels  of  participants  benefited  equally  and  all  categories/levels 
of  knowledge  were  equally  acquired  from  the  gaming  environment. 

Selected  Publications: 

Norton,  J.,  Gonzalez,  G.C.,  Connolly-Gomez,  C.,  Kline,  K.B.  &  Regian,  J.W.  (1995).  Use  of 

gaming  for  intelligent  technical  training  (AL-TP- 1994-0028,  AD  B198812).  Brooks  AFB 
TX:  Armstrong  Faboratory. 

Keywords:  Gaming,  maintenance,  training,  automotive,  troubleshooting 


The  Effect  of  Various  Types  of  Explanation  on  Learning  and  Retention 
Scientists:  Hale  and  Regian 
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Instructors: 


Connolly  and  Robertson-Schiile 


Total  Subjects:  N  =  73 

Purpose  of  Study:  This  study  investigated  issues  surrounding  how  partieipants  learn  to  carry 

out  a  simple  form  of  troubleshooting  using  a  fictional  industrial  system  called 
a  glass  brick  maker  (GBM).  Participants  were  presented  five  diagrams  which 
characterized  the  system.  The  diagrams  were  presented  individually  and 
reflect  a  hierarchy  comprised  of  top-level  system  diagram  followed  by  four 
diagrams  representing  subsystems  (which  decompose  from  the  top  level). 
Based  on  the  treatment  group,  subjects  received  either  explanatory 
information  describing  the  functionality,  mechanism,  or  no  explanation  of 
each  diagram  component.  In  the  learning  phase,  participants  studied  and 
recalled  simple  diagrams.  In  the  troubleshooting  phase,  subjects  were 
required  to  generate  fault  hypotheses  in  response  to  symptoms.  Retention 
tests  were  given  24,  48,  or  96  hours  following  the  learning  phase. 

Results  of  Study:  In  this  study  the  component  effects  model  was  contrasted  with  compilation 
models,  which  assume  that  explanations  will  affect  performance  prior  to  rule 
compilation,  but  not  after  compilation  has  been  accomplished.  Learners 
received  a  rule  learning  phase  immediately  following  system  learning  which 
continued  until  compilation  was  successful.  Then  they  were  given  a  re -test  24, 
48,  or  96  hours  later.  Explanations  affected  performance  in  early  trials  during 
both  immediate  learning  and  re-testing.  This  result  contradicts  predictions  of 
the  compilation  model,  but  supports  the  component  effects  model.  Apparently, 
explanations  are  used  by  learners  only  for  hypothesis  generation,  prior  to 
accrual  of  associative  strength  associated  with  compilation. 

Selected  Publications:  None 

Keywords:  Troubleshooting,  diagnosis,  feedback,  scheduling 


Testing  the  Generalizability  of  the  Effects  of  Massed  vs.  Distributed  Practice 
Scientists:  Goettl,  Yadrick,  Connolly,  and  Regian 

Instructors:  Connolly  and  Robertson-Schiile 

Total  Subjects:  N  =  43 

Purpose  of  Study:  Distributed  practice  is  overwhelmingly  supported  in  the  literature  over 

massed  practice  for  a  variety  of  tasks.  However,  most  of  the  published  studies 
used  simplistic  tasks  and  conducted  the  practice  phase  in  a  matter  of  minutes 
or  hours.  The  issue  as  to  whether  or  not  these  findings  will  generalize  to  real- 
world  training  settings,  where  the  tasks  are  often  complex  and  training  is 
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conducted  over  several  days  to  weeks  is  the  primary  focus  of  this  study.  The 
effeets  of  massing  versus  distributing  praetiee  aeross  40  hours  of  training  were 
investigated  using  three  tasks:  Spaee  Fortress,  Phoenix  and  Math  Word 
Problems  (using  paper  and  pencil).  There  were  two  treatment  groups  that 
were  used  and  randomly  assigned  to  different  weeks;  massed  praetiee  and 
distributed  praetiee.  To  minimize  negative  transfer  between  Spaee  Fortress 
and  Phoenix,  subjeets  were  instrueted  only  to  use  the  keyboard  for  Phoenix. 

In  addition,  to  minimize  any  massing  effeets  during  the  distributed  practice, 
tasks  were  rotated  sueh  that  no  one  task  was  ever  praetieed  sueeessively. 
Retention  and  transfer  tests  always  followed  24  hours  after  praetiee  sessions. 
The  overall  length  of  the  study  was  approximately  40  hours. 

Results  of  Study:  A  signifieant  effect  was  found  for  distributing  practice  Space  Fortress  and 
Math  Word  Problems;  performanee  on  Phoenix  was  not  altered.  This  study 
was  replieated  in  the  new  laboratory  with  Spaee  Fortress  program 
modifieations  being  made. 

Selected  Publications:  None. 

Keywords:  Massed/distributed  practice,  spatial,  problem  solving 


The  Influence  of  Physical  Exercise  on  the  Effects  of  Massed  vs.  Distributed  Practice 
Scientists:  Goettl,  Yadriek,  Connolly,  and  Regian 

Instructors:  Connolly  and  Robertson-Schiile 

Total  Subjects:  N  =  85 

Purpose  of  Study:  The  hypotheses  were  the  same  as  the  earlier  study;  with  the  addition  of  a 
new  variable-exereise.  Again,  the  effects  of  massing  versus  distributing 
praetiee  aeross  40  hours  of  training  were  investigated  using  three  tasks:  Spaee 
Fortress,  Phoenix,  and  Math  Word  Problems  (using  paper  and  pencil). 

Subjects  were  placed  into  one  of  four  conditions:  massed,  massed  with 
exercise,  distributed,  and  distributed  with  exercise.  It  was  hypothesized  that 
exercise  would  offset  possible  fatigue  and  boredom  effects  found  in  massed 
practice  and  improve  performance.  A  pilot  study  examining  different 
exercises  (marching,  walking,  isometrics,  etc.)  revealed  that  a  5-min  march 
increased  heart  rate  on  average  by  25%  and  was  selected  as  the  exercise  to  use 
in  the  study.  The  overall  length  of  the  study  was  approximately  40  hours. 

Results  of  Study:  Results  for  Space  Fortress  and  Math  tasks  showed  a  significant  effect  for 

distributed  training  over  massed  training.  Performance  on  Phoenix  was  not 
altered.  Comparisons  between  the  buildings  for  the  same  treatment  groups 
also  showed  a  significant  effect  for  Space  Fortress  with  the  new  program 
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modifications  being  made.  These  results  help  generalize  traditional  literature 
findings  of  massed  versus  distributed  praetiee  based  on  relatively  simple  tasks 
to  more  eomplex  tasks  whieh  are  typieal  real-world  training  environments.  In 
addition,  this  study  will  help  shed  light  onto  the  faetors  responsible  for  the 
spaeed  praetiee  effeet. 

Selected  Publications: 

Goettl,  B.P.,  Yadriek,  R.M.,  Connolly  Gomez,  C.,  Regian,  J.W.,  &  Shebilske,  W.L.  (1996). 

Alternating  task  modules  in  training  eomplex  tasks.  Human  Factors,  38,  330-346. 

Keywords:  Massed,  spaeed,  distributed,  training,  praetiee,  exereise 


Reducing  the  Time  of  Training  to  Test  the  Effects  of  Massed 
vs.  Distributed  Practice 

Scientists:  Goettl,  Yadriek,  Connolly  Gomez,  and  Regian 

Instructors  Connolly  and  Robertson-Sehiile 

Total  Subjects:  N  =  48 

Purpose  of  Study:  This  experiment  was  based  on  previous  findings,  whieh  indieated  strong 

effeets  for  distributing  over  massing  praetiee  for  three  different  types  of  tasks: 
Spaee  Fortress,  Phoenix  Desktop-Flight  Simulator,  and  Math  Word  Problems 
(using  paper  and  peneil  format).  This  study  mirrored  the  previous  study  in 
terms  of  seheduling,  however,  training  time  was  redueed  by  approximately 
50%  and  exereise  was  not  manipulated.  Subjeets  in  this  study  reeeived 
approximately  5  hours  of  training  on  eaeh  of  the  tasks:  Spaee  Fortress,  Math 
Tutor,  and  Phoenix.  The  sehedule  was  arranged  so  that  those  subjeets  in  the 
distributed  group  would  never  reeeive  more  than  35  minutes  of  eonseeutive 
hours  of  training  on  one  task.  The  massed  group  reeeived  the  same  total 
amount  of  training  on  eaeh  task  as  the  distributed  group;  however,  they 
finished  all  the  training  on  one  task  before  going  on  to  the  next  task.  Posttests 
always  followed  24  hours  after  training.  Again,  to  reduee  possible  negative 
transfer  effeets  between  Spaee  Fortress  and  Phoenix,  subjeets  were  only 
allowed  to  use  the  keyboard,  rather  than  the  joystiek  on  the  Phoenix  task. 

This  study  should  help  us  bridge  our  results  in  the  large  massed  versus 
distributed  study  whieh  eontained  tasks  over  a  one  week  period  to  the 
literature  whieh  used  tasks  within  a  very  short  period  of  time  (usually  only  a 
few  hours).  The  approximate  length  of  the  study  was  20  hours. 

Results  of  Study:  Currently  being  analyzed. 

Selected  Publications:  None. 
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Keywords:  Massed,  spaced,  distributed,  training,  practice,  exercise 


Phoenix:  The  Interaction  Between  Instructional  Modality  and  Gender  (Pilot  Study) 

Scientists:  Regian,  Goettl,  and  Connolly 

Instructors:  Connolly  and  Robertson-Schule 

Total  Subjects:  N  =  10 

Purpose  of  Study:  The  effectiveness  of  instructions  presented  via  text  or  text  and  speech  and 
how  attitudinal  variables  may  interact  with  the  presentation  modality  was 
investigated.  It  was  hypothesized  that  speech  in  addition  to  the  text  may 
provide  a  more  motivational  environment  in  which  to  learn  Phoenix, 
furthermore,  it  was  suggested  that  there  may  be  an  interaction  between  the 
instructional  modality  and  gender.  Specifically,  female  subjects  may  benefit 
more  from  the  text  plus  verbal  instructions  than  do  male  subjects.  The  length 
of  the  experiment  was  approximately  16  hours  spread  across  4  days. 

Results  of  Study:  Not  enough  subjects  (5  per  group)  completed  the  study  to  run  any  meaningful 
ANOVAs.  However,  empirical  analysis  of  the  data  reveals  that  the  speech 
group  performed  worse.  This  effect  may  be  an  artifact  of  the  low  number  of 
subjects  who  finished  the  study,  the  unbalanced  gender  ratio  between 
treatment  groups,  and  differences  in  the  pretest  ability.  The  subjects  with 
speech  tended  to  agree  that:  (a)  the  speech  made  the  task  more  interesting,  (b) 
speech  forced  them  to  look  at  the  displays,  and  (c)  they  learned  more  from  the 
speech  than  if  they  had  just  read  the  screen.  None  of  the  subjects  agreed  with 
the  statement  that  the  speech  was  hard  to  understand.  In  sum,  the  data  we 
have  are  not  conclusive  one  way  or  the  other,  but  if  the  trend  continues  with 
additional  data,  it  would  be  a  very  interesting,  counter-intuitive  finding.  A 
follow-up  study  is  needed  to  verify  these  results. 

Selected  Publications:  None. 

Keywords:  Modality,  training,  synthetic  speech,  spatial,  gender 


Phoenix:  The  Interaction  between  Instructional  Modality 
(Text  vs.  Text  and  Speech)  and  Gender 

Scientists:  Regian,  Goettl,  and  Connolly 

Instructors:  Connolly  and  Robertson-Schule 

Total  Subjects:  N  =  82 
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Purpose  of  Study:  The  effectiveness  of  instructions  presented  via  text  or  text  and  speech  and 
how  attitudinal  variables  may  interact  with  the  presentation  modality  was 
investigated.  It  was  hypothesized  that  speech  in  addition  to  the  text  may 
provide  a  more  motivational  environment  in  which  to  learn  Phoenix.  It  was 
also  hypothesized  that  there  may  be  an  interaction  between  the  instructional 
modality  and  gender.  Specifically,  female  subjects  may  benefit  more  from  the 
text  plus  verbal  instructions  than  do  male  subjects.  The  length  of  the 
experiment  was  approximately  16  hours  spread  across  4  days. 

Results  of  Study:  Initial  analysis  indicates  a  gender  by  treatment  interaction,  with  the  female 
subjects  benefiting  more  than  the  male  subjects  from  the  text  combined  with 
speech  instructional  treatment.  In  addition,  the  subjects  within  the  speech 
treatment  tended  to  agree  that:  (a)  the  speech  made  the  task  more  interesting, 
(b)  speech  forced  them  to  look  at  the  displays,  and  (c)  they  learned  more  from 
the  speech  than  if  they  had  just  read  the  screen.  None  of  the  subjects  agreed 
with  the  statement  that  the  speech  was  hard  to  understand. 

Selected  Publications:  None. 

Keywords:  Modality,  training,  synthetic  speech,  spatial,  gender,  ATI 


Phoenix:  Testing  the  Effectiveness  of  Skill  Acquisition  (Joystick  vs.  Keyboard) 
Scientists:  Goettl  and  Regian 

Instructors:  Linda  Robertson-Schiile 

Total  Subjects:  N  =  28 

Purpose  of  Study:  The  goal  of  this  study  was  to  examine  the  effectiveness  of  skill  acquisition 
for  the  Phoenix  desktop-flight  simulator  using  either  the  joystick  or  keyboard 
only.  Subjects  trained  on  slalom  and  strike  Phoenix  tasks.  The  overall  length 
of  the  study  was  approximately  4  hours,  with  military  recruits  used  as  subjects 
rather  than  civilians. 

Results  of  Study:  The  effects  that  were  obtained  were  largely  practice  effects.  Although  the 

joystick  group  tended  to  make  more  gates  during  training  and  the  posttest,  the 
results  were  generally  not  significant.  However,  there  was  a  significant 
interaction  between  fidelity  and  practice  for  number  of  gates  made  on  the  hard 
courses.  The  keyboard  group  showed  more  improvement  than  did  the  joystick 
group.  The  cause  may  be  the  variability  of  practice  effect.  If  this  were  the 
case  then  we  would  expect  the  joystick  group  to  be  better  on  a  retention  test  or 
a  transfer  test;  which  was  not  measured,  but  rather  a  transfer  test  in  the  strike 
task.  As  in  the  slalom  task,  the  performance  trends  favored  the  keyboard 
group  (shorter  average  time,  more  kills,  shorter  distance,  higher  scores). 
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although  the  effects  were  not  significant.  It  is  possible  that  the  study  was  not 
long  enough  and  that  the  joystick  group  was  still  adjusting  to  the  shifts  from 
joy  to  key  and  back  to  joy  and  the  effect  will  manifest  itself  later  in  training. 
Perhaps  there  was  some  compensatory  rivalry  among  the  subjects.  It  is 
possible  those  subjects  in  the  joystick  condition  were  not  exposed  to  enough 
consistent  mapping  during  training.  The  study  will  be  rerun  to  delineate  these 
factors. 

Selected  Publications:  None. 

Keywords:  Fidelity,  joystick,  keyboard,  spatial,  training 


Drive  Simulator  (Pilot) 


Scientists:  Gugerty  and  Shute 

Instructors:  Connolly  and  Robertson-Schiile 

Total  Subjects:  N  =  70 

Purpose  of  Study:  This  initial  pilot  study  was  to  collect  timing  data,  establish  protocols,  and  to 
debug  software.  To  develop  a  better  understanding  of  "situational  awareness" 
relative  to  the  knowledge  and  cognitive  processes  involved  in  this  complex 
construct.  To  answer  quantitative  questions  regarding  working  memory,  e.g., 
how  many  moving  objects  can  people  monitor  and  how  accurately  can  they 
monitor  them.  Furthermore,  to  answer  some  qualitative  questions,  e.g.,  what 
type  of  strategies  (based  upon  experience,  i.e.,  novice/expert  driver/pilot)  does 
one  implement  when  a  crisis  situation  presents  itself? 

Results  of  Study:  Timing  data,  experimental  protocols,  and  software  problems  noted  for 
correction. 

Selected  Publications:  None 

Keywords:  Driving  Simulator,  Situational  Awareness 


Stat 

Scientists: 
Instructors: 
Total  Subjects: 


Lady  II:  Piloting  tbe  Descriptive  Statistics  Module  of  tbe  Tutor 

Shute 

Connolly  and  Robertson-Schiile 
N  =  80 
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Purpose  of  Study:  The  purpose  of  the  study  was  to  gather  pilot  data  on  the  instructional 

efficacy  for  the  non-student  model  version  of  the  Descriptive  Statistics  Tutor. 

Results  of  Study:  Pilot  data  gathered  led  to  changes  in  the  instruction  and  program  abilities  for 
future  studies. 

Selected  Publications:  None. 

Keywords:  Stat  Lady  Descriptive  Statistics  Tutor,  ITS,  ATI 


System  T^WO:  Tutor  vs.  Traditional  Classroom  Instruction  for  Individuals  vs.  Dyads 
Scientists:  Connolly,  Kline,  and  Regian 

Instructors:  Connolly  and  Robertson-Schule 

Total  Subjects:  N  =  109 

Purpose  of  Study:  Research  conducted  in  the  CoLab  comparing  traditional  versus  tutor-based 

learning  using  Stat  Lady  revealed  no  posttest  differences.  Significant  differences 
were  found  when  test  items  were  categorized  as  declarative  and  procedural 
information.  This  finding  suggests  difference  in  underlying  knowledge  structure 
based  upon  the  instructional  method  used.  In  addition,  research  findings  have 
shown  that  collaborative  learning  is  an  effective,  cost-efficient  method  of 
conducting  training  for  skilled  tasks.  However,  little  is  known  about  the 
effectiveness  of  collaborative  learning  for  other  types  of  tasks  that  are  less  skill 
specific.  The  purpose  of  this  study  was  to  evaluate  the  effectiveness  of  the  tutor 
versus  traditional  classroom  instruction  for  individuals  versus  dyads.  The  study 
consisted  of  approximately  20  hours  distributed  over  five  days.  Post  hoc  analysis 
using  Pathfinder/Multi-Dimensional  Scaling  techniques  (MDS)  will  be  used  to 
evaluate  possible  differences  in  the  underlying  cognitive  structure  of  the  treatment 
groups  as  compared  to  that  of  the  expert. 

Results  of  Study:  Initial  results  indicate  that  the  tutor  group  performed  significantly  better  than  the 
traditional  classroom  group  on  the  posttest.  Specifically,  the  classroom  group 
increased  their  performance  by  1  standard  deviation  (8%)  and  the  tutor  group  by  2 
standard  deviations  (17%)  on  the  posttest.  Additionally,  questionnaire  results 
suggest  that  the  tutor  group  preferred  their  instruction,  thought  that  they  had 
learned  more,  and  were  more  open  to  taking  the  course  again  than  were  the 
classroom  group. 

Selected  Publications: 
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Connolly-Gomez,  C.  (1994).  US  Air  Force  compares  computer  based  training  to  traditional 
classroom  training.  Rappn’  with  Roscoe  Newsletter,  1(1),  p.  2. 

For  more  information  contact:  Cathy  Connolly  at  (941)  275-7720, 
CConnolly@galaxyscientilic.com,  or  Kevin  Kline  at  (210)  671-2667. 

Keywords:  Instructional  method,  collaborative,  pedagogy,  dyad,  intelligent  tutor 


Team  Composition  and  Type  of  Worked  Examples  in  Algebra  Word  Problem  Solving 

Scientists:  Yadrick  and  Regian 

Instructors:  Connolly  and  Robertson-Schule 

Total  Subjects:  N  =  102 

Purpose  of  Study:  The  purpose  of  this  study  was  to  examine  the  simultaneous  and/or 
interactive  effects  of  difficulty  of  instructional  approach  and  team 
composition  of  mathematics  learning.  Subjects  worked  individually  or  in 
teams,  and  were  classified  as  high,  medium,  or  low  ability  (relative  to  the 
subject  population)  and  teamed  with  others  based  on  their  performance  on  the 
pretest  and  a  section  of  the  Armed  Services  Vocational  Aptitude  Battery 
(ASVAB).  There  were  High-High,  Low-Low,  Medium-Medium,  and  High- 
Low  ability  teams.  In  addition,  team  composition  was  crossed  with  task 
complexity  which  was  manipulated  by  presenting  two  different  kinds  of 
worked  example  sets.  The  ‘difficult  set’  consisted  of  two  relatively  simple 
examples,  leaving  subjects  to  figure  out  how  to  adapt  formulas  and  build 
equations  for  more  complex  problems.  The  ‘easy  set’  consisted  of  a  set  of 
comprehensive  examples,  representing  the  full  range  of  problem  difficulty. 
The  study  consisted  of  approximately  16  hours  over  five  days. 

Results  of  Study:  Awaiting  further  data. 

Selected  Publications:  None. 

Keywords:  Problem  solving,  mathematics,  ATI,  example,  collaborative 


Phoenix:  Investigating  the  Optimal  Ratio  of  Practice  to  Observation  for  Learning 
Scientists:  Goettl,  Connolly,  and  Regian 

Instructors:  Connolly  and  Robertson-Schule 

Total  Subjects:  N  =  97 
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Purpose  of  Study:  Results  in  Observational  Phoenix  I  were  similar  to  those  found  in  the 

Spaee  Fortress  dyadie  study.  Speeifically,  results  indieated  that  partieipants 
showed  superior  performance  over  those  in  the  control  condition,  reinforcing 
the  theory  that  a  complex  task  can  be  partially  learned  through  observation. 
Moreover,  comparisons  of  slalom  and  strike  task  performance  revealed  that 
participants  acquired  both  strategic,  as  well  as,  psychomotor  aspects  of  the 
task  through  observation.  This  study  attempted  to  expand  these  findings  and 
investigate  the  optimal  ratios  of  practice  to  observation.  Subjects  either 
performed  100,  75,  50,  25,  or  0  percent  of  the  time  or  observed  a  partner  for 
the  other  part  of  the  time.  All  tests  were  taken  individually.  Based  upon  the 
results  of  this  study,  we  attempted  to  determine  if  there  was  a  basic  underlying 
formula  to  guide  practitioners  in  estimating  how  much  observation  time 
equates  to  "hands-on"  time.  The  length  of  the  task  was  approximately  16 
hours  across  five  days. 

Results  of  Study:  This  study  examined  the  performance  to  observation  ratio  in  training  a 

complex  computer-based  flight  simulator  task.  Subjects  were  divided  into 
five  different  groups.  One  group  performed  100%  of  the  time  during  training, 
the  second  group  performed  75%  of  the  time  and  observed  25%  of  the  time, 
the  third  group  performed  50%  of  the  time  and  observed  50%  of  the  time,  the 
fourth  group  performed  25%  of  the  time  and  observed  75%,  and  the  fifth 
group  observed  100%  of  the  time.  Based  on  previous  observational  learning 
literature,  we  predicted  a  linear  relationship  between  performance  and  percent 
of  time  performing.  Results  however,  suggest  a  non-linear  relationship 
between  performance  and  percent  of  time  performing.  Performance  was 
slightly  better  when  a  combination  of  performance  and  observation  was  used 
rather  than  performance  alone.  These  results  indicate  that  observational 
learning  plays  an  important,  yet  often  neglected  role  in  learning  complex 
computer-based  tasks  and  suggests  that  small  group  oriented  computer-based 
training  systems  may  be  more  instructionally  and  cost  effective  than 
individually  oriented  computer-based  training  systems. 

Selected  Publications: 

Gomez,  C.C.,  &  Goettl,  B.  P.  (1995).  Investigating  tradeoffs  between  practice  and  observation 
in  automated  instruction.  Proceedings  of  the  39th  Annual  Meeting  of  the  Human  Factors 
&  Ergonomics  Society  (1340-1344).  Santa  Monica,  CA:  The  Human  Factors  & 
Ergonomics  Society. 
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Investigating  Three  Theories  of  Symptom  Elaboration 


Scientists:  Hale  and  Regian 

Instructors:  Connolly  and  Robertson-Schiile 

Total  Subjects:  N  =  60 

Purpose  of  Study:  Using  the  fietional  industrial  system  ealled  a  glass  briek  maker  (GBM)  this 
experiment  attempted  to  explore  the  role  of  symptom  elaborations  on  the 
search  component  of  symptom-fault  learning.  Two  hypotheses  were 
evaluated:  (1)  symptom  elaboration  improves  search  performance  through 
topographic  indexing,  if  original  system  learning  is  organized  topographically, 
(2)  symptom  elaboration  improves  search  performance  through  malfunction 
indexing  if  original  system  learning  is  organized  functionally.  A  third 
hypothesis  is  that  symptom  elaboration  improves  search,  independently  of  the 
nature  of  original  learning. 

Results  of  Study:  Two  measures  were  of  interest  in  this  experiment:  (1)  percent  correct  at  each 
trial  of  rule  learning  and  (2)  trials  to  criterion.  The  first  measure  represents  an 
absolute  measure  of  accuracy  and  allows  evaluation  of  the  locus  of  the 
elaboration  effect  (hypothesis  generation  versus  hypothesis  recognition).  The 
second  measure  allows  an  evaluation  of  the  effect  of  symptom  elaboration  on 
learning  rate.  The  percent  correct  results  demonstrated  a  strong  interaction 
between  the  type  of  information  available  during  original  learning  and  the 
nature  of  symptom  elaboration  during  rule  learning.  Results  showed  that  any 
type  of  symptom  elaborations  improved  learning  rates  following  original 
learning  based  on  topography.  Furthermore,  there  was  no  difference  between 
topographic  and  malfunction  symptom  elaborations.  Topographic  elaborations 
significantly  improved  performance,  topographic  elaborations  significantly 
degraded  performance.  These  results  suggest  that  providing  learners  with 
elaborative  information,  at  the  point  of  hypothesis  generation,  that  allows 
them  to  construct  causal  chains  from  symptoms  to  the  most  plausible  faults 
will  greatly  enhance  learning  by  minimizing  the  number  of  errors  committed 
during  the  early  stages  of  rule  construction. 

Selected  Publications: 

Hale,  C.R.  (November,  1994).  Explanations  improve  the  search  component  of  symptom-fault 
rule  learning.  Annual  meeting  of  the  Psychonomic  Society. 

Hale,  C.R.  (October,  1994).  Training  implications  of  differential  elaboration  in  learning  to 
troubleshoot  devices.  Annual  meeting  of  the  Human  Factors  and  Ergonomics  Society. 

Keywords:  Troubleshooting,  diagnosis,  feedback,  elaboration 
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Comparing  Two  Software  Versions  (Pilot  Study) 


Scientists:  Hale  and  Regian 

Instructors:  Connolly  and  Robertson-Schiile 

Total  Subjects:  N  =  1 1 

Purpose  of  Study:  Using  Topo  (version  7)  software  as  a  comparison,  a  newly  developed 
version  of  the  Topo  software  (version  8)  was  piloted  to  discern  any 
differences  between  the  software  programs.  Any  programmatic  type  errors  or 
faults  will  be  changed  and  the  newer  version  of  the  program  will  be  used  in 
the  Topo  6  experiment. 

Results  of  Study:  Results  revealed  that  the  newly  developed  version  eight  of  the  Topo  software 
did  not  contain  any  programmatic  type  errors  and  was  more  flexible,  reduced 
the  amount  of  time  and  effort  to  load  the  software  and  decreased  the 
typographical  and  entry  errors  made  by  the  test  administrators. 

Selected  Publications:  Awaiting  follow-up  study. 

Keywords:  Software,  comparisons,  Topo 


Stat  Lady  II:  Piloting  tbe  Data  Organization  and  Plotting  Module  (DS-1)  Criterion  Tasks 

Scientist:  Shute 

Instructors:  Connolly,  Gawlick-Grendell,  and  Robertson-Schiile 

Total  Subjects:  N  =  79 

Purpose  of  Study:  The  four  forms  of  the  Data  Organization  and  Plotting  Module  (DS-1) 
Criterion  Tests  (i.e..  Forms  A,  B,  C,  D)  were  pilot-tested  for  inter-item 
reliability.  This  pilot  was  used  to  determine  whether  items  assessing  the  same 
Curricular  Element  (CE)  were  correlated  with  each  other.  Test  item  numbers 
are  the  same  across  the  four  forms  (e.g.,  item  7  assessed  CE-3)  and,  in  the 
absence  of  instruction,  subjects  should  perform  similarly  across  matched  / 
parallel  items.  Problems  in  forms  A  and  B  were  designed  to  be  isomorphic,  in 
that  they  share  similar  formats.  Questions  in  forms  C  and  D  were  designed  to 
be  different  in  format  than  questions  in  A  and  B,  yet  isomorphic  to  one 
another.  Subjects  were  assigned  randomly  to  one  of  eight  test  orders  that 
resulted  from  a  fully  counterbalanced  design.  The  eight  orders  were:  ACBD, 
ADCB,  BDAC,  BCDA,  CBAD,  CADB,  DBCA,  DABC. 
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Results  of  Study:  Data  analyses  showed  that  the  following  eombinations  of  tests  are  to  be  used 
in  any  upeoming  experiments:  A/D  and  B/C.  Thus,  half  the  subjeets  per  study 
will  reeeive  forms  A  &  D  as  their  pretest  and  forms  B  &  C  as  their  posttest. 
The  other  half  will  reeeive  forms  B  &  C  as  their  pretest  and  forms  A  &  D  as 
their  posttest. 

Selected  Publications: 

Shute,  V.J.  (1995).  SMART:  Student  modeling  approach  for  responsive  tutoring.  User 

Modeling  and  User-Adapted  Instruction  (UMUAI).  Special  issue  on  “Student  Modeling,” 
(5)  1-44. 

Keywords:  Stat  Lady  Descriptive  Statistics  Tutor,  criterion  tests,  interitem  reliability 


Stat  Lady  II:  Testing  the  Validity  of  Stat  Lady  as  a  Computer-Based  Tutor 

Scientist:  Shute 

Instructors:  Connolly,  Gawlick-Grendell,  and  Robertson-Schiile 

Total  Subjects:  N  =  72 

Purpose  of  Study:  The  purpose  of  the  study  was  to  determine  how  well  subjects  learned  from 
this  version  of  the  tutor,  and  test  the  predictive  validity  of  the  computed  p(CE) 
values.  In  addition,  this  pilot  was  used  to  repair  any  bugs  in  the  program  and 
acquire  an  estimate  of  learning  times.  Subjects  were  administered  the  non¬ 
student  model  version  of  the  Descriptive  Statistics  Module  1  (DS-1)  with 
remediation  and  the  mastery  criterion  disabled  (i.e.,  all  subjects  received  only 
2  problems  to  solve  per  problem  set).  Although  the  student  model  (SMART) 
was  in  place  and  updating  itself  (i.e.,  computing  all  p(CE)’s),  it  did  not  affect 
the  course  of  the  curriculum.  Instruction  time  was  fixed  to  a  maximum  of  3.5 
hours.  Subjects  were  also  administered  a  demographic  questionnaire  and  an 
affective  survey  to  examine  opinions  about  the  tests,  the  tutor,  and  how  well 
they  believed  they  learned  Descriptive  Statistics. 

Results  of  Study:  Results  showed  about  a  67%  increase  from  Pretest  to  Posttest  scores.  Subjects 
appeared  to  learn  very  well  from  the  tutor,  and  this  becomes  the  score  to  beat 
in  subsequent  experiments  involving  the  student  model.  Stepwise  regression 
equations  were  computed  predicting  posttest  from  pretest,  tutor  data  (i.e., 
p(CE)’s),  and  other  demographic  data  (e.g.,  number  of  math  courses,  attitude 
about  statistics,  gender,  education).  In  all  instances,  the  tutor  data  was  the 
strongest  predictor  of  outcome,  accounting  for  34  to  63.4%  of  unique 
variance.  Analyses  of  the  affective  survey  revealed  that  subjects  really 
enjoyed  the  instruction.  Initially,  most  felt  frustrated  by  the  on-line  pretest, 
but  after  exposure  to  the  tutor,  they  reported  that  they  mostly  felt  in  control  of 
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their  learning,  the  tutor’s  feedbaek  was  helpful,  and  that  they  learned 
something.  Based  on  the  results  of  this  pilot,  the  deeision  was  made  to  allow 
more  time  on  the  tutor  (i.e.  ,>3.5  hours)  for  the  formal  studies  (i.e..  Studies  C 
and  D).  Numerous  ehanges  were  also  made  to  the  program  to  elarify 
instruction  and  improve  functionality. 

Selected  Publications: 

Shute,  V.J.  (1995).  SMART;  Student  modeling  approach  for  responsive  tutoring.  User 

Modeling  and  User-Adapted  Instruction  (UMUAI).  Special  issue  on  “Student  Modeling,” 
(5)  1-44. 

Keywords:  Stat  Lady  Descriptive  Statistics  Tutor,  predictive  validity,  microadaptation, 

affect 


Investigating  how  Students  Learn  Troubleshooting  Skills 

Scientist:  Hale 

Instructors:  Connolly,  Gawlick-Grendell  and  Robertson-Schiile 

Total  Subjects:  N  =  5 1 

Purpose  of  Study:  This  study  investigated  issues  surrounding  how  subjects  learn  to  carry  out 
a  simple  form  of  troubleshooting  using  a  fictional  industrial  system  called  a 
glass  brick  maker  (GBM)  with  the  newest  version  of  Topo  (version  8). 
Subjects  were  presented  five  diagrams  which  characterize  the  system.  The 
diagrams  were  presented  individually  and  reflect  a  hierarchy  comprised  of 
top-level  system  diagrams  followed  by  four  diagrams  representing  subsystems 
(which  decompose  from  the  top  level).  Based  on  the  treatment  group,  subjects 
received  either  explanatory  information  describing  the  functionality, 
describing  the  mechanism,  or  no  explanation  of  each  diagram  component.  In 
the  learning  phase,  subjects  studied  and  recalled  simple  diagrams.  In  the 
troubleshooting  phase,  subjects  were  required  to  generate  fault  hypotheses  in 
response  to  symptoms.  Retention  tests  were  given  24,  48,  or  96  hours 
following  the  learning  phase. 

Results  of  Study:  Currently  being  analyzed. 

Selected  Publications:  None. 

Keywords:  Software,  comparisons,  Topo 
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ATM  2  --  Task  differences  in  Alternating  Task  Modules 


Scientists:  Yadrick,  Goettl,  Connolly,  Regian,  and  Shebilske 

Instructors:  Connolly  and  Robertson-Schiile 

Total  Subjects:  N  =  85 

Purpose  of  Study:  Tbe  purpose  of  this  study  was  to  see  whether  the  similarity  of  the  tasks 
influeneed  the  magnitude  of  the  ATM  effect.  This  was  the  first  of  at  least 
three  projected  experiments  intended  to  examine  the  limits  of  Alternating 
Task  Modules  in  massed  vs.  spaced  training.  Previous  studies  established  that 
ATMs  produce  sizable  spacing  effects  when  there  are  three  dissimilar  tasks, 
one  reasonable  and  important  empirical  question  concerns  how  dissimilar 
tasks  need  to  be  in  order  to  produce  spacing  effects.  If  for  example, 
alternating  Space  Fortress  with  Phoenix  produces  spacing  effects,  will 
alternating  Space  Fortress  with  Space  Fortress  keyboard  produce  spacing 
effects?  This  question  has  practical  implications.  For  example,  in  a  military 
technical  school  it  may  be  impractical  to  train  by  alternating  two  or  three 
different  subjects  matters,  since  the  purpose  of  the  school  is  to  train  a  single 
subject  matter.  As  another  example,  in  industry  it  may  be  difficult  to  train  a 
given  group  of  employees  by  alternating  two  or  three  completely  different 
training  topics.  However,  in  either  case  it  might  be  possible  to  train  by 
alternating  different  aspects  or  areas  of  a  topic  or  subject  matter.  In  this  study 
we  will  compare  a  group  that  alternates  between  Space  Fortress  and  Phoenix 
with  a  group  that  alternates  between  Space  Fortress  and  Space  Fortress- 
Keyboard.  About  15  subjects  will  be  randomly  assigned  to  each  of  6  groups. 

Results  of  Study:  Currently  being  analyzed. 

Selected  Publicatious: 

Goettl,  B.  P.,  Yadrick,  R.M.,  Connolly-Gomez,  Regian,  J.  W.,  &  Shebilske,  W.L.  Alternating 
task  modules  in  isochronal  training  of  complex  tasks.  Human  Factors,  38(2),  330-346. 

Keywords:  Alphabet  Arithmetic,  Phoenix,  Automaticity,  Spatial,  Quantitative 


The  Effects  of  Exercise  on  the  Spacing  Effect  (A  Control  Group) 
Scientists:  Goettl,  Yadrick,  and  Connolly 

Instructors:  Connolly  and  Robertson-Schiile 

Total  Subjects:  N  =  76 
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Purpose  of  Study:  The  purpose  of  this  study  was  to  investigate  the  effeets  of  exereise  on  the 
spaeing  effeet  using  three  different  tasks:  Spaee  Fortress,  Math  tutor,  and 
Phoenix.  A  previous  study  suggested  that  exereise  reduees  overall 
performanee  and  size  of  the  practiee  effeet.  In  that  study,  exereise  was 
subtraeted  from  the  total  practiee  time.  This  study  represented  the  comparison 
group  for  the  exercise  condition  in  the  previous  study.  Participants  repeated 
the  massed  and  distributed  schedules  with  exercise  from  the  original  massed 
versus  distributed  experiment.  However,  instead  of  exercising  these  subjects, 
they  merely  rested  during  the  time  allotted  for  exercise.  Participants  received 
10  hours  of  training  on  Space  Fortress,  16  hours  on  the  Math  Tutor,  and  10 
hours  on  Phoenix.  There  were  30  participants  per  group,  two  groups  drawn 
from  the  temporary  pool.  About  30  participants  were  randomly  assigned  to 
each  group  with  the  constraint  that  the  maleifemale  ratio  was  constant  across 
groups.  The  experiment  took  approximately  40  hours  per  week  to  complete. 
The  most  recent  versions  of  Space  Fortress,  Math  Tutor,  and  Phoenix  were 
used. 

Results  of  Study:  Preliminary  results  indicated  that  the  alternating  task  modules  resulted  in 

better  performance  on  SF  acquisition  and  retention.  In  addition,  the  exercise 
condition  produced  worse  performance  on  the  SF  transfer  task.  Results  for 
Algebra  word  problems  and  Phoenix  are  still  being  examined. 

Selected  Publications: 

Goettl,  B.  P.,  Shebilske,  W.L.,  Connolly-Gomez,  C.,  &  Robertson-Shule,  L.  (1996).  The  effects 
of  exercise  and  alternating  task  modules  in  skill  acquisition.  Proceedings  of  the  Human 
Factors  and  Ergonomics  Society  40**^  Annual  Meeting,  2,  1097-1101. 

Keywords:  Spatial,  observation,  cognitive  strategies,  modeling 


Stat  Lady  II  —  Descriptive  Statistics  Module  (Computer-Based  Tutor) 
Study  D:  Predictive  Validity  of  SMART 


Scientist:  Shute 

Instructors:  Connolly,  Gawlick-Grendell,  and  Robertson-Schule 

Total  Subjects:  N  =  104 

Purpose  of  Study:  The  purpose  of  this  study  was  to  test  the  efficacy  of  the  Stat  Lady  tutor  and  the 
predictive  validity  of  SMART.  In  addition,  comparisons  between  this  study  and 
Study  C  (microadaptive  version  of  Stat  Lady)  were  made  to  evaluate  the 
remediation  component  (i.e.,  learning  gains,  time  on  tutor).  Subjects  were 
administered  the  non-student  model  version  of  the  Descriptive  Statistics  Module  1 
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(DS-1)  with  remediation  and  the  mastery  eriterion  disabled  (i.e.,  all  subjects  were 
limited  to  completing  exactly  2  problems  per  problem  set).  Although,  the  student 
model  (SMART)  was  in  place  and  updating  itself  (i.e.,  computing  all  p(CE)’s),  it 
did  not  affect  the  course  of  the  curriculum.  Subjects  were  also  administered  a 
demographic  questionnaire  and  an  Affective  Survey  (to  examine  opinions  about 
the  tests,  the  tutor,  and  how  well  they  believed  they  learned  Descriptive 
Statistics).  In  addition,  cognitive  and  personality  measures  were  administered  and 
the  Big  Five  Inventory  was  administered  at  the  beginning  and  end  of  testing  to 
measure  test-retest  reliability. 

Results  of  Study:  Pretest  to  posttest  gains  were  found  to  be  2.2  standard  deviations  (Pre  M  =  44%, 
SD  =  14;  Post  M  =  75%,  SD  =  15).  Stepwise  regression  equations  were  computed 
predicting  posttest  from  pretest,  tutor  data  (i.e.,  p(CE)’s),  an  aptitude  factor  score, 
education  (years  in  school),  and  gender.  The  tutor  data  were  found  to  be  the  first 
(and  strongest)  predictor  of  outcome,  accounting  for  54%  of  unique  variance. 

Next  to  enter  the  equation  was  aptitude,  accounting  for  an  additional  1 1%  of 
unique  outcome  variance.  Analyses  of  the  Affective  Survey  showed  that  initially, 
subjects  were  frustrated  with  the  on-line  pretest.  After  exposure  to  the  tutor,  they 
reported  that  they  liked  the  tutor,  enjoyed  their  learning  experience,  felt  the  tutor 
feedback  was  helpful,  and  felt  in  control  of  their  learning.  Preliminary 
comparisons  with  Study  C  data  showed  that  learning  efficiency  was  greater  using 
the  non-intelligent  version  of  Stat  Eady,  but  acquisition  was  significantly  higher 
using  the  intelligent  version. 

Selected  Publications: 

Shute,  V.J.  (1995).  SMART:  Student  modeling  approach  for  responsive  tutoring.  User  Modeling 
and  User-Adapted  Instruction  (UMUAI).  Special  issue  on  “Student  Modeling,”  (5)  1-44. 

Shute,  V.J.  &  Gluck,  K.  A.  (1995,  April).  Artifacts  of  learning:  The  Stat  Lady  experience.  Paper 
presented  at  the  annual  meeting  of  American  Educational  Research  Association,  San 
Francisco,  CA  (April,  1995). 

Keywords:  Stat  Eady  Descriptive  Statistics  Tutor,  microadaptation,  macroadaptation,  affect. 

Cognitive  Abilities  Measures  (CAM),  Big  Five  Personality  Inventory 


Stat  Lady  II  —  Descriptive  Statistics  Module  (Computer-Based  Tutor) 

Pilot  C:  Microadaptive  Functioning  of  SMART  (Debugging  and  Preliminary  Results) 


Scientist: 


Shute 


Instructors:  Connolly,  Gawlick-Grendell,  and  Robertson-Schiile 

Total  Subjects:  N  =  32 
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Purpose  of  Study:  The  purpose  of  this  pilot  was  to  test  the  functionality  of  SMART  with 

mastery  criterion,  microadaptation,  and  remediation  in  place.  In  addition,  this 
pilot  was  used  to  gain  an  estimate  of  tutor  completion  times  with  this  version 
of  the  Stat  Lady  tutor  and  to  test  the  stability  of  the  program.  Results  from 
this  pilot  were  used  to  determine  an  acceptable  level  for  mastery.  Subjects 
had  to  attain  mastery  of  all  Curricular  Elements  (CE’s)  before  moving  past 
any  section  of  the  curriculum.  Otherwise,  remediation  was  invoked. 
Instruction  time  was  not  fixed,  to  allow  subjects  all  the  time  necessary  to 
attain  mastery  of  the  curriculum  (up  to  2  full  days).  Subjects  first  took  two 
on-line  pretests,  then  received  adaptive  instruction;  followed  by  two  posttests. 
Subjects  were  also  administered  a  demographic  questionnaire  and  an 
Affective  Survey  (to  examine  opinions  about  the  tests,  the  tutor,  and  how  well 
they  believed  they  learned  Descriptive  Statistics).  In  addition,  cognitive  and 
personality  measures  were  assessed  using  a  quantitative  subset  of  the  CAM 
4.0  battery  and  the  Big  Eive  Personality  Inventory. 

Results  of  Study:  The  first  group  of  subjects  participating  in  this  pilot  was  used  to  test  various 
mastery  criterion  levels  (i.e.,  0.5,  0.6,  0.7,  0.8).  It  was  found  that  0.7  was  the 
optimal  level  because  the  others  made  learning  too  easy  or  too  hard.  The 
decision  rule  used  to  determine  when  a  student  is  remediated  was  also 
modified  (based  on  whether/how  often  the  student  had  been  remediated  on 
that  CE  previously).  Based  on  the  results  of  this  pilot,  the  decision  was  made 
to  allow  more  time  on  the  tutor  (i.e.,  up  to  3  days  rather  than  2  days).  Changes 
were  also  made  to  repair  or  improve  the  functionality  of  the  program. 

Selected  Publications: 

Shute,  V.J.  (1995).  SMART;  Student  modeling  approach  for  responsive  tutoring.  User 

Modeling  and  User- Adapted  Instruction  (UMUAI).  Special  issue  on  “Student  Modeling,” 
(5)  1-44. 

Keywords:  Stat  Eady  Descriptive  Statistics  Tutor,  mastery  learning,  microadaptation, 

affect,  remediation.  Cognitive  Abilities  Measures  (CAM  4.0),  Big  Eive 
Personality  Inventory 


Assessing  the  Reliability  of  the  Algebra  Word  Problem  Solving  (AWP)  Pre/Post  Tests 
Scientists:  Yadrick  ,  Regian,  Connolly,  and  Robertson-Schule 

Instructors:  Connolly,  Gawlick-Grendell,  and  Robertson-Schule 

Total  Subjects:  N  =  39 

Purpose  of  Study:  This  study  was  conducted  to  assess  the  reliability  of  the  (AWP)  pre/post 
tests  in  response  to  reviewers'  comments  regarding  a  paper  submitted  for 
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publication.  Paper-and-pencil  tests  were  eounterbalanced  and  administered  in 
a  2-hour  slot  maximum,  whieh  was  eonsistent  with  the  original  study. 


Results  of  Study:  Results  revealed  that  the  reliability  of  two  forms  of  the  math  pretest  and 
posttest  were  parallel. 

Selected  Publications: 

Yadrick,  R.M.,  Robertson-Sehule,  L.,  &  Connolly-Gomez,  C.  (1996).  Interface,  instruetional 
approaeh,  and  domain  learning  with  a  mathematies  problem-solving  environment. 
Computers  in  Human  Behavior,  12(4),  527-548. 

Yadriek,  R.M.,  Regian,  J.W.,  Connolly-Gomez,  C.,  &  Robertson-Sehule,  L.  (1997).  Dyadie  vs. 
individual  practiee  with  exploratory  and  direetive  mathematies  tutors.  Journal  of 
Educational  Computing  Research,  17(2),  165-186. 

Yadriek,  R.M.,  Regian,  J.W.,  Connolly-Gomez,  C.,  &  Robertson-Sehule,  L.  (1995).  Individual 
and  eooperative  group  learning  with  user-eontrolled  and  program-eontrolled  mathematies 
tutors.  (AL/HR-TP- 1995 -0031,  ADA303591).  Brooks  AFB  TX;  Armstrong  Laboratory, 
Human  Resourees  Directorate. 

Yadriek,  R.M.,  Regian,  J.W.,  Robertson-Sehule,  L.  &  Connolly  Gomez,  C.  (1995).  Interfaee, 
instruetional  approaeh,  and  domain  learning  with  a  mathematics  problem-solving 
environment.  (AL/HR-TP-1995-0032,  ADA303561).  Brooks  AFB  TX:  Armstrong 
Laboratory,  Human  Resources  Direetorate. 

Keywords:  Algebra  word  problems,  quantitative,  reliability 


Testing  the  Number  of  Trials  to  Achieve  Automaticity  Utilizing 
the  Phoenix  and  Alphabet  Arithmetic  Tasks 

Scientists:  Yadriek,  Shebilske,  and  Goettl 

Instructors:  Connolly,  Gawliek-Grendell,  and  Robertson-Schiile 

Total  Subjects:  N  =  47 

Purpose  of  Study:  These  pilots  were  eondueted  to  prepare  for  the  preliminary  ATM  4  study. 

Subjeets  were  given  a  series  of  Alphabet  Arithmetie  and  Phoenix  trials  in 
order  to  estimate  the  total  number  of  trials  whieh  eould  be  completed  within 
the  time  eonstraints  and  to  work  out  proeedures.  Subjeets  were  tested  for  a 
maximum  of  4  hours  on  one  day. 
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Results  of  Study:  The  results  showed  that  21  trials/sessions  were  suffieient  for  subjects  to 
automatize  the  various  problem  sets  within  a  4-hour  time  slot. 

Selected  Publications:  None.  Awaiting  follow-up  study. 

Keywords:  Alphabet  Arithmetic,  Phoenix,  automaticity,  spatial,  quantitative 


Space  Fortress:  Assessing  Gender  Differences  with  Spatial  Tasks 

Scientists:  Regian  and  Shute 

Instructors:  Connolly,  Gawlick-Grendell,  and  Robertson-Schule 

Total  Subjects:  N  =  60 

Purpose  of  Study:  Initially,  a  pilot  study  was  run  to  obtain  timing  data  and  to  test  various 
potential  stress  options.  Subjects  were  trained  on  the  Space  Fortress  task 
across  two  days  to  assess  spatial  gender  differences  during  training. 

Results  of  Study:  Results  found  that  sound  (i.e.,  subjects  listening  to  white  noise  while  playing 
Space  Fortress)  did  not  impact  upon  subjects’  performance.  An  uncertainty 
variable  was  identified  to  be  used  in  subsequent  analyses  within  Gender 
experiment  #1 .  This  was  the  pseudo-option  version  of  Space  Fortress  (i.e.,  Ss 
were  told  that  the  Space  Fortress  version  had  changed,  when  actually  it  had 
not.).  Also,  there  was  no  main  effect  found  in  the  efficacy  of  group- 
discussions;  in  relation  to  improving  female  performance. 

Selected  Publications 

Regian,  J.W.  &  Shute,  V.J.  (1994).  Understanding  and  reducing  gender  differences  in  task 
performance.  Human  Factors  and  Ergonomics  Society  Technical  Training  Group 
Newsletter,  Spring  (1),  6-7. 

Keywords:  Gender  differences,  spatial,  psychomotor 


Scientist: 


Stat  Lady  II  —  Descriptive  Statistics  Module  (Computer-Based  Tutor) 
Study  C:  Microadaptive  Functioning  of  SMART 


Shute 


Instructors:  Connolly,  Gawlick-Grendell,  and  Robertson-Schule 

Total  Subjects:  N  =  107 
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Purpose  of  Study:  The  purpose  of  this  study  was  to  test  the  efficacy  of  SMART  with  mastery 

criterion,  microadaptation,  and  remediation  in  place.  Mastery  was  set  at  0.70,  and 
subjects  had  to  attain  mastery  of  relevant  CE’s  (i.e.,  those  below  the  pre-set 
mastery  criterion)  before  moving  on  to  the  next  section  of  the  curriculum. 
Otherwise,  remediation  was  invoked.  Instruction  time  was  not  fixed,  allowing 
subjects  to  attain  mastery  of  the  curriculum  (up  to  3  full  days)  at  their  own  pace. 
Subjects  first  took  the  on-line  pretest,  then  adaptive  instruction,  followed  by  the 
two  posttests.  Subjects  also  completed  a  demographic  questionnaire  and  an 
Affective  Survey  (to  examine  opinions  about  the  tests,  the  tutor,  and  how  well 
they  believed  they  learned  Descriptive  Statistics).  In  addition,  cognitive  and 
personality  measures  were  assessed  using  a  quantitative  subset  of  the  CAM  4.0 
battery  and  the  Big  Five  Personality  Inventory. 

Results  of  Study:  Evaluating  the  remediation  component  involved  comparing  the  relative  degree  of 
learning  with  this  feature  turned  on  (Study  C)  versus  when  it’s  not  employed 
(Study  D).  Preliminary  results  of  Study  C  showed  significant  gains  in  pretest  to 
posttest  learning  (Pre  M  =  50%;  Post  M  =  82%).  Due  to  incoming  differences  on 
the  pretest  between  Study  C  and  Study  D  data,  an  ANOVA  was  computed  on 
posttest  using  pretest  as  a  covariate.  Significant  differences  between  groups  were 
found,  with  higher  performance  levels  being  exhibited  by  Study  C  subjects.  Thus, 
having  the  microadaptive  features  in  place  (i.e.,  remediation  and  mastery 
criterion)  does  help  learners  acquire  greater  degrees  of  domain-specific 
knowledge.  In  terms  of  learning  efficiency.  Study  C  subjects  required 
significantly  more  time  to  complete  the  tutor  (Study  C  M  =  7.6  hours;  Study  D  M 
=  4.4  hours).  Thus,  although  the  microadaptive  instruction  significantly  increased 
acquisition,  it  also  significantly  added  to  learning  time.  Other  analyses  from  these 
data  are  forthcoming. 

Selected  Publications: 

Shute,  V.J.  (1995).  SMART;  Student  modeling  approach  for  responsive  tutoring.  User  Modeling 
and  User-Adapted  Instruction  (UMUAI).  Special  issue  on  “Student  Modeling,”  (5)  1-44. 

Shute,  V.J.  &  Gluck,  K.  A.  (1995,  April).  Artifacts  of  learning:  The  Stat  Lady  experience.  Paper 
presented  at  the  annual  meeting  of  American  Educational  Research  Association,  San 
Francisco,  CA  (April,  1995). 

Keywords:  Stat  Eady  Descriptive  Statistics  Tutor,  mastery  learning,  microadaptation,  affect, 

remediation.  Cognitive  Abilities  Measures  (CAM  4.0),  Big  Five  Personality 
Inventory 

Early  vs.  Late  Spacing  of  Training  and  its  Effect  on  Automaticity 

Scientists:  Yadrick,  Shebilske,  and  Goettl 
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Instructors: 


Connolly,  Gawlick-Grendell,  and  Robertson-Schiile 


Total  Subjects:  N  =  90 

Purpose  of  Study:  The  purpose  of  the  study  was  to  determine  effeets  of  early  or  late  spaeing  of 
training  in  an  automaticity  paradigm.  One  hypothesis  was  that  massing 
training  may  be  beneficial  at  particular  points  in  the  learning  process.  The 
purpose  of  this  study  was  to  follow  acquisition  of  a  new  skill  from  its 
introduction  until  subjects  have  automatized  the  skill,  using  one  of  the  four 
training  schedules;  either  massed  training  throughout,  an  ATM  regimen  early 
in  training  followed  by  massed  training,  or  an  ATM  regimen  late  in  training 
following  massed  training.  Two  tasks  were  involved;  Phoenix  and  an 
"Alphabet  Arithmetic"  task  previously  studied  by  Logan  &  Klapp  (1991).  In 
alphabet  arithmetic,  subjects  verily  correct  or  incorrect  equations.  For 
example:  A  +  2  =  C,  meaning  that  C  is  the  second  letter  past  A  in  the 
alphabet.  Logan  &  Klapp's  work  using  this  paradigm  supports  their  retrieval 
theory  of  automatization,  wherein  automatization  is  the  process  of  converting 
from  algorithmically  calculating  the  answer  for  each  problem  to  retrieving  the 
answers  directly  from  memory.  Thus,  this  paradigm  will  allow  differentiation 
of  the  effects  of  massing  vs.  ATM  on  different  phases  of  performance.  There 
was  no  test  per  se,  because  identifying  an  effect  involves  comparing  reaction 
time  and  slope  data  for  each  session  and  group,  to  the  corresponding  data  for 
each  other  session  and  group.  We  decided  upon  a  filler  task,  Phoenix,  which 
is  a  desktop-flight  simulator.  Subjects  were  tested  once  a  week  for  a  day, 
maximum  4-hour  slot  minimum. 

Results  of  Study:  Currently  being  analyzed. 

Selected  Publications:  None. 

Keywords:  Alphabet  Arithmetic,  Phoenix,  automaticity,  spatial,  quantitative 


Space  Fortress:  The  Effects  of  Gender,  Testosterone  Levels, 

Group  Discussion,  and  Stress  on  Performance 

Scientists:  Regian  and  Shute 

Instructors:  Gawlick-Grendell,  Robertson-Schiile,  Snooks,  and  Robbins 

Total  Subjects:  N  =  130 

Purpose  of  Study:  The  purpose  of  this  study  was  to  investigate  differences  in  training 

performance  using  Space  Fortress  (a  complex  spatial  task)  with  regard  to 
gender,  testosterone  levels,  group  discussion,  and  stress  conditions.  For 
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Gender  Study  experiment  #1,  subjects  were  trained  for  approximately  16 
hours  on  the  Space  Fortress  task  (2-8  hour  days).  They  completed  nine 
training  sessions  on  the  Space  Fortress  task  with  five  15-minute  discussion 
groups  within  training  sessions.  The  Control  group  worked  on  a  paper-and- 
pencil  anagram  task.  Subjects  that  were  tested  had  not  played  more  than  20 
hours  per  week  of  either  home  computer  or  video  arcade  games.  In  addition, 
they  qualified  on  the  Space  Fortress  Aiming  Task  (i.e.,  total  score  of  780  or 
more  on  at  least  one  of  three  games).  A  signed  consent  form  was  obtained 
from  the  subjects  for  their  sample  of  saliva.  For  Gender  experiment  #4,  all 
subjects  worked  independently  and  completed  four  Space  Fortress  training 
sessions  with  the  induced  stress.  Three  stress  conditions  were  used:  (1)  real 
options  (i.e.,  more  difficult  SF  version  with  subjects  having  to  determine  these 
changes  on  their  own),  (2)  pseudo-options  (i.e.,  subjects  were  the  SF  version 
had  changed,  when  actually  it  had  not),  and  (3)  normal  (i.e.,  subjects 
continued  training  with  normal  version  of  SF  and  were  told  nothing). 
Additionally,  saliva  samples  were  collected  from  each  subject  to  evaluate 
testosterone  levels. 

Results  of  Study:  Preliminary  findings  show  that  the  relationship  between  testosterone  levels 
and  spatial  performance  were  supported  along  with  other  findings  in  the 
literature  concerning  a  significant  quadratic  trend. 

Selected  Publications:  None. 

Keywords:  Gender  differences,  spatial,  psychomotor 


Stat  Lady  II  —  Descriptive  Statistics  Module  (Computer-Based  Tutor) 

Stat  Lady  E:  Replication  Study:  Differential  Practice  Effects  on  Retention 

Scientists:  Shute  and  Gluck 

Instructors:  Connolly,  Gawlick-Grendell,  Robertson-Schiile,  Snooks,  and  Robbins 

Total  Subjects:  N  =  305 

Purpose  of  Study:  This  replication  study  consisted  of  five  practice  conditions  (between- 

subjects  design)  in  DS-1:  (a)  AA  (N  =  80),  (b)  AE  (N  =  50),  (c)  EA  (N  = 
50),  (d)  EE  (N  =  80),  and  (e)  Eeamer’s  Choice  (N  =  80):  (where  A  = 
Abbreviated,  E  =  Extended,  and  EC  =  learners  control  number  of  problems 
they  solve).  Subjects  in  the  Abbreviated  condition  solved  one  problem  per 
problem  set  and  in  the  Extended  condition  they  solved  four  problems  per  set. 
The  mixed  groups  switched  over  after  problem  set  3  (3/5  =  .6),  as  the  original 
retention  study  switched  mixed  Ss  after  .61  of  the  curriculum  was  completed 
(14/23).  Across  five  problem  sets,  the  total  number  of  problems  to  solve  per 
condition  were:  AA  (5),  AE  (11),  EA  (14),  EE  (20),  EC  (between  5-20).  All 
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subjects  were  within  the  age  range  of  18-30  and  had  no  prior  Statistics  courses 
in  their  educational  backgrounds.  Subjects  first  took  an  on-line  pretest, 
received  identical  instruction  from  Stat  Lady  (between  1-3  days),  then  took  an 
on-line  posttest.  All  subjects  will  be  recalled  to  participate  in  a  follow-up 
study  (6  months  later),  where  they’ll  take  a  parallel  test  for  assessment  of 
retention  (approximately  8  hours).  Then  the  impact  on  learning  (from  self- 
selecting  practice  opportunities)  will  be  assessed  provided  a  projected 
minimum  of  33%  of  all  subjects  to  return. 

Results  of  Study:  Preliminary  results  showed  that  subjects  assigned  to  the  EE  condition  scored 
higher  on  their  posttests  in  comparison  to  the  other  conditions.  This  was 
probably  due  to  greater  practice  opportunities  and  problem-solving  time. 
However,  there  were  no  differences  with  regard  to  extracurricular  activities, 

(i.e.,  personal  review  of  instructed  material,  accessing  the  on-line  dictionary, 
and/or  reviewing  the  hypertext  reference  guide  defining  symbols  and 
formulas).  Interestingly,  results  showed  that  subjects  assigned  to  the  EC 
condition  not  only  learned  the  most  from  the  Statistics  tutor  (reflected  in 
obtaining  the  highest  outcome  scores),  but  they  also  acquired  relevant 
knowledge  and  skills  in  significantly  less  time  than  the  other  conditions.  This 
was  apparent  for  both  their  instructional  and  problem-solving  times  relative  to 
all  other  conditions. 

Selected  Publications: 

Shute,  V.J.  (1995).  SMART;  Student  modeling  approach  for  responsive  tutoring.  User  Modeling 
and  User- Adapted  Instruction  (UMUAI).  Special  issue  on  “Student  Modeling,”  (5)  1-44. 

Shute,  V.J.  &  Gluck,  K.  A.  (1995,  April).  Artifacts  of  learning:  The  Stat  Lady  experience.  Paper 
presented  at  the  annual  meeting  of  American  Educational  Research  Association,  San 
Erancisco,  CA. 

Keywords:  Stat  Eady  Descriptive  Statistics  Tutor,  microadaptation,  macroadaptation,  affect. 

Cognitive  Abilities  Measures  (CAM),  Big  Eive  Personality  Inventory 


Phoenix:  Gender  Differences  Due  to  Spacing  or  Observational  Effects? 
Scientists:  Goettl  and  Connolly 

Instructors:  Gawlick-Grendell,  Robertson-Schiile,  Snooks,  and  Robbins 

Total  Subjects:  N  =  105 

Purpose  of  Study:  Our  previous  studies  showed  that  subjects  benefited  from  observing  a 

Phoenix  task  that  has  significant  cognitive  processing  demands.  In  that  study 
females,  not  males,  showed  an  observational  learning  effect.  In  a  second 
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study,  we  found  a  direct  relationship  between  the  performance  to  observance 
ratio  and  performance,  but  that  performance  was  best  for  a  ratio  of  75/25.  In 
that  study  both  males  and  females  benefited  from  observational  learning.  The 
goal  of  this  particular  study  was  to  further  explore  possible  gender  differences 
of  observational  learning  and  to  disentangle  spacing  effects  from  observation 
effects.  Subjects  were  placed  into  one  of  five  training  groups.  All  groups 
received  training  on  the  Phoenix  strike  task  in  one  of  the  following  schedules: 
(1)  Spaced  Control  -  alternate  every  two  trials  between  Phoenix  and  filler  task. 
The  filler  task  used  was  Windows  Solitaire.  (2)  Spaced  Observation  -  alternate 
every  two  trials  between  perform  and  observe.  (3)  Massed  Control  -  block  of 
filler  task  followed  by  block  of  Phoenix.  (4)  Massed  P-0  -  block  of 
performing  Phoenix,  block  of  observing  Phoenix.  (5)  Massed  0-P  -  block  of 
observing  Phoenix,  block  of  performing  Phoenix.  Results  of  the  first  two 
studies  were  promising.  They  showed  support  for  Caroll  and  Bandura’s 
model  of  observational  learning.  However,  this  study  is  needed  to  test 
specific  control  conditions  to  confirm  our  hypotheses.  The  design  addressed 
the  question  of  whether  observation  should  be  intermixed  with  opportunities 
to  perform.  In  addition,  it  addressed  the  question  of  whether  early  observation 
is  more  important  than  late  observation.  Also,  it  was  of  interest  to  examine 
spatial  and  quantitative  individual  differences  in  observational  learning,  so 
CAM  and  VS  AT  batteries  were  obtained.  The  study  took  2  days  to  complete. 

Results  of  Study:  Currently  being  analyzed. 

Selected  Publications:  None. 

Keywords:  Gender  differences.  Observational  learning,  spacing,  spatial,  psychomotor 

Assessing  Individual  Differences  Using  Individual  Goals 

Scientists:  Regian,  Greenis,  and  Goettl 

Instructors:  Gawlick-Grendell  and  Robertson-Schule 

Total  Subjects:  N  =  73 

Purpose  of  Study:  The  purpose  of  the  study  was  to  investigate  individual  differences  by 

assigning  individualized  goals  to  subjects.  Results  are  expected  to  indicate 
ways  in  which  goals  may  be  incorporated  into  complex  task  training  and 
Intelligent  Tutoring  Systems  to  maximize  learning.  This  study  serves  as  a 
predecessor  to  a  forthcoming  study,  which  will  investigate  the  beneficial 
effects  of  process  feedback  and  continuously  updated,  individualized  goals  on 
complex  task  performance.  Research  has  shown  that  goal  setting  consistently 
leads  to  higher  levels  of  performance  on  simple  tasks.  However,  on  complex 
tasks,  setting  goals  does  not  result  in  increased  performance  and  may  result  in 
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performance  decrements.  The  goal  of  this  study  was  to  investigate  one  way  in 
which  goal  setting  may  augment  complex  task  performance,  through  the 
assignment  of  individualized  goals.  Subjects  were  placed  into  one  of  three 
training  groups:  (1)  Control  group,  (i.e.,  their  feedback  read  “do  your  best”), 
(2)  Standard  Goal,  (i.e.,  subjects  were  given  a  difficult,  specific  performance 
goal,  which  was  based  on  the  average  performance  of  individuals  in  the 
control  group,  and  (3)  Individualized  Goal,  (i.e.,  subjects  were  assigned  a 
goal  which  was  individually  tailored  to  their  initial  performance  and  aptitudes. 
All  subjects  were  between  the  ages  18  and  30,  had  an  educational  minimum  of 
a  high  school  or  general  equivalency  diploma  (GED),  and  had  no  prior  job  or 
experience  with  Air  Traffic  Control.  Each  subject  performed  2  practice  trials 
and  nine  performance  trials  (3  blocks  of  3-  10  minute  trials).  Goals  were 
updated  before  each  new  block  of  trials  (every  30  minutes).  Individualized 
goals  are  expected  to  facilitate  performance  by  increasing  self-efficacy  and 
reducing  negative  affect  and  strategy  switching. 

Results  of  Study:  Currently  being  analyzed. 

Selected  Publications:  None. 

Keywords:  Spatial,  Air  Traffic  Controller,  varied  goal  feedback 


Space  Fortress:  Examining  Top-Down  Processing 

Scientists:  Shebilske  and  Goettl 

Instructors:  Gawlick-Grendell,  Robertson-Schiile,  Snooks,  and  Robbins 

Total  Subjects:  N  =  36 

Purpose  of  Study:  The  purpose  was  to  examine  the  role  of  top-down  processing  during 

intercession  intervals  in  the  distributed  practice  effect  that  has  been  observed 
during  the  acquisition  of  Space  Portress.  We  are  specifically  testing  two 
hypotheses.  Pirst,  we  predicted  that  elaboration  would  be  more  effective  later 
in  skill  acquisition  when  resource  load  has  been  reduced.  Second,  we 
predicted  that  the  benefits  of  elaboration  would  be  obtained  primarily  by 
subjects  having  a  high  elaborative  cognitive  style.  To  test  these  predictions 
we  ran  four  different  groups:  (1)  Massed  Total  Emphasis,  (2)  Massed 
Multiple  Emphasis  on  Components,  (3)  Elaboration  Total  Emphasis,  and  (4) 
Elaboration  Multiple  Emphasis  on  Components.  Each  group  was  divided  into 
high  and  low  elaborators  using  a  median-split  technique.  We  predicted  an 
interaction  such  that  high-elaborators  would  benefit  the  most  from  the 
elaboration.  This  experiment  provided  an  important  signpost  for  our 
programmatic  research  on  spacing  effects  and  on  observational  learning  in 
group  protocols.  The  connection  to  observational  learning  was  through  the 
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hypothesis  that  it  also  facilitated  learning  through  top-down  processing.  All 
subjects  had  no  prior  exposure  to  other  Space  Fortress  experiments. 

Results  of  Study:  Currently  being  analyzed. 

Selected  Publications:  None. 

Keywords:  Elaborations  and  Individual  Differences 

Phoenix  Observational  Learning  III 

Scientist(s):  Barry  P.  Goettl  &  Cathy  Connolly-Gomez 

Instructors/Proctor(s):  {Instructor/Proctor  Name(s)} 

Total  Subjects:  N  =  98 

Purpose  of  Study:  The  goal  of  this  study  was  to  explore  whether  massed  or  spaced 

observation  was  effective  and  when  was  the  best  time  to  apply  observation 
in  training.  Five  groups  were  compared  using  a  desktop  flight  simulator 
task.  Massed-perform-observe  (MPO)  group  performed  a  block  of  massed 
trials,  while  massed-observe-perform  (MOP)  group  observed.  MPO  and 
MOP  groups  switched  their  functions  after  one  hour.  The  participants  in 
spaced-observe  (SO)  group  alternated  between  practice  and  observation 
every  two  trials.  These  three  groups  were  compared  to  massed-  and 
spaced-control  conditions.  Massed  observation  was  expected  to  be  more 
effective  because  it  provides  participants  with  more  opportunities  to 
develop  a  mental  representation  of  the  task  prior  to  the  actual  task 
performance. 

Results  of  Study:  In  contrast  to  the  spaced  observation  group,  participants  in  the  massed 

observation  group  performed  the  task  better  than  those  in  the  control 
conditions.  Also,  participants  in  the  massed  performance  group  performed 
the  task  better  than  participants  in  the  massed-control  condition.  These 
results  suggest  that  observational  learning  is  beneficial  for  both,  the 
observer  and  performer. 

Selected  Publications: 

Goettl,  B.  P.,  &  Connolly-Gomez,  C.  (1997).  The  effects  of  observational  learning  and  practice 
schedule  on  the  acquisition  of  a  complex  skill.  Proceedings  of  the  Human  Factors  and 
Ergonomics  Society  Annual  Meeting  (1180  -  1184).  Santa  Monica,  CA;  The  Human 
Factors  &  Ergonomics  Society. 
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Keywords:  Phoenix,  observational  learning,  massed  and  spaced  practice  schedule,  complex 

skill  acquisition 


Air  Traffic  Control  1  -  Hold  One 


Scientist(s):  Barry  P.  Goettl,  F.  L.  Lee  and  J.R.  Anderson 

Instructors/Proctor(s):  Michael  Anthony 


Total  Subjects:  N  =  1 1 1 

Purpose  of  Study:  The  goal  of  this  experiment  was  to  examine  the  relationship  between 

aptitude,  strategy,  and  performance  in  the  Kanfer-Ackerman  Air  Traffic 
Control  (ATC)  task  as  the  use  of  strategy  changes  with  the  change  in  ATC 
task  environment.  Manipulated  variables  were  the  taxiing  time  of  the 
planes  on  runways  and  the  penalty  for  using  lower  holds  strategy  versus 
no  such  penalty.  In  contrast  to  long  taxiing  time,  short  taxiing  time  and  no 
penalty  condition  were  expected  to  increase  the  utility  of  Hold  1  strategy. 


Results  of  Study:  As  predicted  the  use  of  strategy  changed  as  the  implicit  and  explicit 

changes  in  the  ATC  task  environment  altered.  A  strong  positive 
relationship  was  found  between  aptitude  and  performance.  Also,  a  strong 
relationship  was  found  between  keying  rate  and  performance,  while  no 
penalty  condition  showed  a  strong  negative  relationship  between  Hold  1 
strategy  and  performance.  Most  of  the  strong  relationship  between 
cognitive  ability  and  performance  was  through  the  relationship  between 
ability  and  Hold  1  strategy. 


Selected  Publications: 


Lee,  F.  L.,  Anderson,  J.  R.,  &  Goettl,  B.  P.  (1997).  The  relationship  between  aptitude  and 
strategy  use  in  performing  dynamic,  real-time  tasks.  Available  in 
http://act-r.psv.cmu.edu/publications/pubinfo 

Keywords:  Air  Traffic  Control  task,  adaptive  strategy,  aptitude  and  performance 


Space  Fortress  Dyad-VEC 

Scientist(s):  Barry  P.  Goettl,  Michael  Anthony,  and  Wayne  L.  Shebilske 

Instructors/Proctor(s):  {Instructor/Proctor  Name(s)} 

Total  Subjects:  N  =  168 
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Purpose  of  Study:  The  goal  of  this  study  was  to  create  more  efficient  and  effective  protocol 
by  combining  dyadic  protocol  (individual  vs.  dyad)  and  emphasis  change 
manipulation  (component  vs.  total  score)  on  the  Space  Fortress  computer 
task.  Also  examined  were  the  effects  of  gender  and  interaction  anxiety. 
Moreover,  the  relationship  between  performance  on  the  explicit  and 
implicit  task  components  and  consequent  verbalizable  knowledge  (IE 
questionnaire)  was  examined. 


Results  of  Study:  The  results  suggested  that  participants  in  dyads  improved  more  on  ship 

control  than  individuals;  women  benefited  more  from  emphasis  change 
than  men  and  performance  of  individuals  high  in  interaction  anxiety  was 
inhibited  when  they  were  in  the  dyad-VEC  condition.  Moreover, 
performance  on  explicit  task  components  was  positively  related  to  the 
consciousness  ratings  of  explicit  components  while  performance  on 
implicit  tasks  and  consciousness  ratings  of  implicit  components  showed 
near-zero  correlations. 


Selected  Publications: 

Goettl,  B.  P.,  Anthony,  M.,  Derek,  K.,  Snooks,  S.,  &  Shebilske,  W.  E.  (1998).  Emphasis,  gender 
and  interaction  anxiety  in  individual  and  dyadic  training  protocols.  Proceedings  of  the 
Human  Eactors  and  Ergonomics  Society  42”^^  Annual  Meeting  ( 1412-1416).  Santa 
Monica,  CA;  The  Human  Eactors  &  Ergonomics  Society. 

Goettl,  B.  P.,  Anthony,  M.,  Derek,  K.,  &  Shebilske,  W.  E.  (1998).  Consciousness  ratings 

correlate  with  explicit  but  not  implicit  components  of  a  complex  task.  Paper  accepted  for 
presentation  at  the  39*  Annual  Meeting  of  the  Psychonomics  Society. 

Keywords:  Space  Portress,  variable  emphasis  on  components,  explicit/implicit  dissociation 


Air  Traffic  Control  2  -  Individual  Differences 
Scientist(s):  Barry  P.  Goettl  and  Christian  Schunn 

Instructors/Proctor(s):  Michael  Anthony 
Total  Subjects:  N  =  57 

Purpose  of  Study:  The  goal  of  this  experiment  was  to  introduce  another  approach  to 

individual  differences  in  cognitive  skill  which  states  that  people  differ  in 
how  well  they  can  adaptively  change  their  strategies  as  a  result  of 
changing  features  of  the  task  environment.  The  task  used  in  this 
experiment  was  Kanfer- Ackerman  Air  Traffic  Control  task.  Also,  CAM-4, 
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a  cognitive  assessment  battery  was  used  to  determine  what  cognitive 
abilities  are  assoeiated  with  adaptiveness. 

Results  of  Study:  The  results  indicated  that  most  partieipants  adapted  their  ehoiee  of  strategy 

in  response  to  the  alterations  in  the  task  environment.  One  of  the  eognitive 
sub-skills,  induetive  reasoning  skill,  was  assoeiated  with  adaptiveness  in 
the  general  eontext  of  the  ATC  task  and  variable  manipulations  used  in 
this  experiment. 


Selected  Publications: 

Reder,  L.  M.,  &  Sehunn,  C.  D.  (1998).  Bringing  together  the  psyehometrie  and  strategy  worlds: 
Predieting  adaptivity  in  a  dynamie  task.  In  D.  Gopher  and  A.  Koriat  (Eds.),  Attention  and 
Performanee  XVII:  Cognitive  regulation  of  performanee:  Interaetion  of  theory  and 
applieation.  Cambridge,  MA:  MIT  Press. 

Keywords:  ATC  task,  individual  differences,  adaptive  strategies 


Aircraft  Identification:  Orientation  Constancy  in  the  Picture  Plane  I 


Scientist(s):  Ashworth  and  Robbins 

Proctor(s):  Robbins 


Total  Subjects:  N  =  67 

Purpose  of  Study:  To  quantify  the  normalization  function  associated  with  identifieation  of 
military  aircraft  that  are  rotated  in  the  picture  plane.  To  quantify  the 
normalization  differences  as  a  function  of  overall  visual  similarity  of  the 
stimulus  set. 


Results  of  Study:  Line  drawings  of  current  military  aircraft  of  both  American  and  Foreign 

origin  served  as  stimuli.  Similarity  rating  of  pair-wise  presented  aircraft 
and  card  sorting  tasks  were  used  to  cluster  aircraft  into  highly  similar 
(homogeneous)  or  highly  distinctive  (heterogeneous)  stimulus  sets.  Both 
stimulus  sets  were  presented  to  subjects  in  an  RSVP  identification 
paradigm  wherein  the  subjects  first  learned  to  associate  individual  planes 
with  individual  response  keys,  and  then  were  instructed  to  identify  planes 
that  were  systematically  rotated  within  the  picture  plane.  Resulting 
orientation  normalization  functions  resembled  classical  "mental  rotation" 
curves,  with  homogeneous  stimuli  producing  curves  of  significantly  high 
slope  than  heterogeneous  stimuli.  Results  are  interpreted  to  support  the 
hypothesis  that  highly  homogeneous  stimuli  force  the  object  recognition 
system  to  use  quantitative  features  for  identification  (which  are  more 
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orientation  specific)  while  highly  heterogeneous  stimuli  force  the  use  of 
more  qualitative  features  (which  are  less  orientation  specific). 

Selected  Publications: 

Ashworth,  A.R.S.  Ill,  &  Robbins,  R.  D.  (1996).  Visual  homogeneity  affects  orientation 
sensitivity.  The  37th  Annual  Meeting  of  the  Psychonomic  Society,  Chicago,  IL. 

Keywords:  Aircraft  Identification,  Orientation  Constancy,  Mental  Rotation 

Aircraft  Identification:  Orientation  Constancy  in  the  Picture  Plane  II 

Scientist(s):  Ashworth  and  Robbins 

Proctor(s):  Robbins  and  Snooks 

Total  Subjects:  N  =  81 

Purpose  of  Study:  Replicates  and  extends  the  results  of  Orientation  Constancy  in  the  Picture 
Plane  I  to  three-dimensional  realistically  rendered  military  aircraft. 

Results  of  Study:  Three-dimensional  realistically  rendered  stimuli  of  current  military  aircraft 

of  both  American  and  Foreign  origin  served  as  stimuli.  Similarity  rating  of 
pair-wise  presented  aircraft  were  used  to  cluster  aircraft  into  highly  similar 
(homogeneous)  or  highly  distinctive  (heterogeneous)  stimulus  sets.  Both 
stimulus  sets  were  presented  to  subjects  in  an  RSVP  identification 
paradigm  wherein  the  subjects  first  learned  to  associate  individual  planes 
with  individual  response  keys,  and  then  were  instructed  to  identify  planes 
that  were  systematically  rotated  within  the  picture  plane.  Resulting 
orientation  normalization  functions  resembled  classical  "mental  rotation" 
curves,  with  homogeneous  stimuli  producing  curves  of  significantly  high 
slope  than  heterogeneous  stimuli.  Results  are  interpreted  to  support  the 
hypothesis  that  highly  homogeneous  stimuli  force  the  object  recognition 
system  to  use  quantitative  features  for  identification  (which  are  more 
orientation  specific)  while  highly  heterogeneous  stimuli  force  the  use  of 
more  qualitative  features  (which  are  less  orientation  specific). 

Selected  Publications: 

Ashworth,  A.R.S.  Ill,  &  Robbins,  R.  D.  (1996).  Maintaining  object  constancy  for  recognition  of 
aircraft.  The  3rd  Annual  Recognition  Materials  Conference.  Langley  AFB  VA:  480th 
Intelligence  Group,  36  IS/Recognition  Materials  Branch. 

Keywords:  Aircraft  Identification,  Orientation  Constancy,  Mental  Rotation 
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Space  Fortress  -  MFC  Plus  Rules 


Scientist(s):  Barry  P.  Goettl,  Shirley  F.  Snooks,  and  Wayne  L.  Shebilske 

Instructors/Proctor(s):  {Instruetor/Proetor  Name(s)} 

Total  Subjects:  N  =  233 

Purpose  of  Study:  The  goal  of  this  study  was  to  examine  the  relationship  between  multiple 

emphasis  on  components  (MFC)  and  verbal  elaboration.  The  participants 
were  trained  on  the  Space  Fortress  task.  The  assumption  was  that  verbal 
elaboration  would  focus  participants  on  the  explicit  processes  while  MFC 
would  focus  participants  on  the  implicit  processes.  Four  training 
conditions  were  developed;  MFC  and  TOT  (total  score  emphasis)  with 
and  without  verbal  elaboration. 

Results  of  Study:  The  results  showed  that  MFC  protocol  was  more  beneficial  for  the  low- 

aptitude  learners  while  TOT  protocol  was  more  beneficial  for  the  high- 
aptitude  learners.  The  results  on  the  component  scores  showed  that 
combination  of  MFC  and  verbal  elaboration  increased  the  scores  on  speed 
(explicit  component)  but  decreased  the  scores  on  control  (implicit) 
component. 

Selected  Publications: 

Goettl,  B.  P.,  Snooks,  S.  F.,  &  Shebilske,  W.  F.  (1997).  Fmphasis  change  and  verbal  elaboration 
in  skill  acquisition;  A  tale  of  two  components.  Proceedings  of  the  Human  Factors  and 
Frgonomics  Society  Annual  Meeting  (1195-1199).  Santa  Monica,  CA;  The  Human 
Factors  &  Frgonomics  Society. 

Keywords:  Space  Fortress,  multiple  emphasis  on  components,  verbal  elaboration 

Space  Fortress  Observational  Learning  I 

Scientist(s):  Barry  P.  Goettl  and  Julie  Stiles-Shipley 

Instructors/Proctor(s):  {Instructor/Proctor  Name(s)} 

Total  Subjects:  N  =  264 

Purpose  of  Study:  The  goal  of  these  studies  was  to  look  at  the  effects  of  observation  (no 

observation,  novice  model  and  experienced  model  observation),  model 
skill  level  (novice  and  experienced)  and  practice  schedule  (massed  vs. 
spaced)  on  the  acquisition  of  a  complex  computer-based  task  (Space 
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Fortress).  The  dependent  measure  was  the  average  total  point  seores  for 
test  sessions. 

Results  of  Study:  The  results  suggested  that  there  was  a  slight  advantage  for  observing 

experieneed  models  at  the  beginning  of  training  although  later  in  training 
there  was  no  differenee  between  experieneed  and  noviee  model 
observation.  Both,  noviee  and  experieneed  model  observation  proved  to  be 
more  effective  than  no-observation  condition.  Moreover,  males  only 
seemed  to  have  benefited  more  from  observation  of  either  novice  or 
experienced  models  .  Also,  males  showed  a  greater  benefit  of  spaced 
training  schedule  than  females. 

Selected  Publications: 

Goettl,  B.  P.,  Stiles-Shipley,  J.,  Derek,  K.,  Jimenez,  L.,  &  Hardcastle,  K.  (1998).  Observing 
expert  and  novice  models  in  acquisition  of  complex  skills.  Proceedings  of  the  16^*^ 
Applied  Behavioral  Sciences  Symposium  (121-125).  US  Air  Force  Academy,  Colorado 
Springs,  CO. 

Stiles-Shipley,  J.  A.,  Goettl,  B.  P.,  Connolly,  C.,  Jimenez,  L.  (1998).  The  influence  of  expert  and 
novice  models  on  acquisition  of  a  complex  skill.  Proceedings  of  the  Human  Factors  and 
Ergonomics  Society  42”^^  Annual  Meeting  (1641).  Santa  Monica,  CA:  The  Human 
Factors  &  Ergonomics  Society. 

Keywords:  observational  learning,  practice  schedules,  human  performance 


Temporal  Reasoning  Study  (pilot) 


Scientist(s):  Eisa  Torreano 

Instructors/Proctor(s):  Michael  Anthony,  Shirley  Snooks 

Total  Subjects:  N  =  36 

Purpose  of  Study:  This  pilot  is  an  initial  investigation  of  how  people  make  inferences  that 

depend  upon  temporal  information,  specifically  the  order  of  events 
occurring  over  time.  After  reading  a  four-sentence  description  about 
everyday  events,  participants  are  posed  a  question  concerning  the  order  of 
the  occurrence  of  two  of  the  events  described  in  the  scenario.  The  purpose 
of  this  pilot  is  to  (a)  measure  the  reading  times  for  the  premises  and 
questions  for  each  scenario,  (b)  record  the  accuracy  of  the  inferences 
made,  and  (c)  assess  the  types  of  incorrect  inferences  that  are  made. 
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Results  of  Study:  No  conclusions  have  been  drawn  from  this  data-set  due  to  (a)  intermittent 

failure  of  the  programs  to  run  properly,  and  (b)  the  diffieulty  that  some  of 
the  partieipants  had  in  doing  the  task;  further  analysis  is  required. 

Selected  Publications:  N/A  (pilot  study) 

Keywords:  temporal  reasoning 


Validating  Alternative  Pedagogical  Approaches  to  Teaching  Basic  Algebra  Skills 

(Math!) 

Scientist(s):  Ross  Willis 

Instructors/Proctor(s):  Michael  Anthony,  Kristie  Hardcastle,  Leticia  Jiminez 

Total  Subjects:  N  =  40 

Purpose  of  Study:  This  study  was  a  pilot  study  designed  to  test  two  novel  pedagogical 

approaches  to  teaehing  students  how  to  solve  basie  algebraie  equations 
and  word  problems.  The  first  graphieal  pedagogy  is  an  analogy  that  likens 
solving  an  algebraic  equation  to  maintaining  balance  on  a  seesaw.  Then 
solving  algebraie  equations,  one  must  be  careful  to  perform  an  operation 
to  both  sides  of  the  equation  in  order  to  maintain  equality,  or  balanee.  A 
seesaw  presented  beneath  an  equation  tilts  when  an  operation  has  been 
performed  to  one  side  of  the  equation  and  not  to  the  opposite  side  of  the 
equation.  The  seeond  graphical  pedagogy  is  a  “highlighter”  method  that 
suceessively  highlights  keywords  in  a  word  problem  and  dynamically 
builds  an  algebraic  equation  representing  the  situation  deseribed  in  the 
word  problem. 

Results  of  Study:  Learners  who  used  the  seesaw  pedagogy  did  not  have  higher  posttest 

scores  (controlling  for  pretest  seore)  than  learners  who  merely  practiced 
solving  algebraie  equations,  in  the  absence  of  instruction.  However, 
learners  who  used  the  highlighter  pedagogy  in  the  word  problems  lesson 
achieved  significantly  higher  posttest  seores  (controlling  for  pretest  seore) 
than  participants  assigned  to  the  praetiee  condition. 

Selected  Publications:  N/A  (pilot  study) 

Keywords:  intelligent  tutoring  systems,  algebra  instruetion 
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A  Comparison  of  Four  Methods  of  Remediating  Students’  Errors 
in  a  Computer-based  Instructional  System 

(Math3) 

Scientist(s):  Ross  Willis 

Instructors/Proctor(s):  Michael  Anthony,  Kristie  Hardcastle,  Leticia  Jiminez,  John  Billimek, 

Eduardo,  Henry 

Total  Subjects:  N  =  169 

Purpose  of  Study:  Computer-based  instruetion  may  address  a  student’s  remedial  needs  along 
two  dimensions:  re-teaching  format  and  remedial  elements.  In  terms  of  re¬ 
teaching  format,  the  computer-based  instructional  system  ean  re-teaeh  the 
deficient  knowledge  and  skills  using  the  initial  instructional  screens  in  the 
form  a  review  of  the  original  material.  Alternatively,  the  system  ean 
present  a  set  of  new  instructional  screens  based  on  an  alternative 
pedagogieal  strategy  (as  deseribed  in  Math2).  In  terms  of  remedial 
elements,  the  instructional  system  may  present  a  complete  set  of 
instruetional  sereens  designed  to  address  all  elements  of  the  eurrieulum,  as 
in  a  complete  review  of  the  initial  instruetional  materials.  Alternatively, 
the  system  may  diagnose  the  student’s  error  and  select  a  set  of 
instructional  screens  speeifieally  designed  to  address  the  student’s 
apparent  misconception.  This  experiment  examined  the  four  remediation 
methods  and,  additionally  assessed  individual  differenees  with  regard  to 
the  four  remediation  methods.  The  principal  research  question  was:  Which 
form(s)  of  remediation  branching  is  (are)  best  for  whieh  leamer(s)? 

Results  of  Study:  Results  from  Experiment  3  did  not  support  the  notion  of  aptitude- 

treatment  interactions  using  the  manipulations  deseribed  previously. 
Overall,  learners  with  high  working-memory  proficiency  achieved  higher 
posttest  scores  and  required  less  time  to  complete  the  lessons  than  learners 
with  low  working-memory  profieiency. 

Selected  Publications:  None. 

Keywords:  intelligent  tutoring  systems,  remediation,  eognitive  diagnosis,  aptitude-treatment 

interactions 
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GK  Comprehension 


Scientist(s):  Lisa  Torreano 

Instructors/Proctor(s):  Kristie  Hardcastle,  Henry  Avila 

Total  Subjects:  N  =  72 

Purpose  of  Study:  This  study  is  an  initial  step  towards  determining  whether  there  are  general 

eognitive  abilities  (e.g.,  that  are  identifiable  w/a  subset  of  the  CAM  tests) 
that  eorrelate  with  patterns  of  proeessing  metaphorieal  uses  of  verbs. 

Results  of  Study: 

Selected  Publications:  None. 

Keywords:  figurative  language,  metaphor  proeessing 


Building  a  Bug  Library 


Scientist(s):  Ross  Willis 

Instructors/Proctor(s):  Ross  Willis 


Total  Subjects:  N  =  28 

Purpose  of  Study:  When  an  instruetional  system  utilizes  eognitive  diagnosis  to  determine  a 

student’s  speeifie  miseoneeption,  a  bug  library  must  be  eonstrueted.  This 
was  aeeomplished  by  having  28  partieipants  eomplete  a  paper-and-peneil 
algebra  quiz  eonsisting  of  34  problems.  Partieipants  were  instrueted  to 
explieitly  show  all  work  used  to  eompute  answers. 


Results  of  Study:  I  analyzed  the  written  work  of  eaeh  partieipant  who  obtained  ineorreet 

responses  for  problems.  Doing  so,  I  was  able  to  reeognize  several 
ineorreet  solution  paths  that  were  followed  by  one  or  more  partieipants. 
These  data  were  used  in  the  final  version  of  the  instruetional  system. 


Selected  Publications: 


Willis,  R.  E.,  &  Taraban,  R.  (submitted).  A  problem-solving  approaeh  to  analyzing  students’ 
algebra  errors.  Submitted  to  the  Journal  of  Developmental  Edueation. 

Keywords:  intelligent  tutoring  systems,  eognitive  diagnosis,  bug  library 
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